-- Logs begin at Thu 2022-05-19 11:16:47 UTC, end at Thu 2022-05-19 12:50:38 UTC. -- May 19 12:00:45.283584 dv-cinder-778193-27-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 12:00:45.294146 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 12:00:45 2022] *** May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: nodename: dv-cinder-778193-27-iscsi May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: machine: x86_64 May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: clock source: unix May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: pcre jit disabled May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: detected number of CPU cores: 4 May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: current working directory: / May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: detected binary path: /usr/bin/uwsgi-core May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: your processes number limit is 47680 May 19 12:00:45.296162 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: your memory page size is 4096 bytes May 19 12:00:45.297641 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: detected max file descriptor number: 2048 May 19 12:00:45.297641 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: lock engine: pthread robust mutexes May 19 12:00:45.297641 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: thunder lock: enabled May 19 12:00:45.297641 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 12:00:45.297641 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 12:00:45.343617 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: Python main interpreter initialized at 0x55d6dea7f9c0 May 19 12:00:45.343617 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: python threads support enabled May 19 12:00:45.343617 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: your server socket listen backlog is limited to 100 connections May 19 12:00:45.343617 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: your mercy for graceful operations on workers is 90 seconds May 19 12:00:45.343981 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: mapped 403077 bytes (393 KB) for 2 cores May 19 12:00:45.344058 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: *** Operational MODE: preforking *** May 19 12:00:45.344318 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: *** uWSGI is running in multiple interpreter mode *** May 19 12:00:45.344318 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: spawned uWSGI master process (pid: 80591) May 19 12:00:45.344528 dv-cinder-778193-27-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 12:00:45.345953 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: spawned uWSGI worker 1 (pid: 80603, cores: 1) May 19 12:00:45.346047 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: spawned uWSGI worker 2 (pid: 80604, cores: 1) May 19 12:00:45.347617 dv-cinder-778193-27-iscsi devstack@keystone.service[80591]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 12:00:51.244502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 12:00:51.244502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: from cryptography.utils import int_from_bytes May 19 12:00:51.245314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 12:00:51.245314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: from cryptography.utils import int_from_bytes May 19 12:00:51.286221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 12:00:51.286221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: from cryptography.utils import int_from_bytes May 19 12:00:51.286861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 12:00:51.286861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: from cryptography.utils import int_from_bytes May 19 12:00:51.296884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 12:00:51.301945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 12:00:51.301945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] command line args: [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 12:00:51.301945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 12:00:51.301945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ================================================================================ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 12:00:51.301945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] admin_token = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.301945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] config_dir = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.301945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.301945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] config_source = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.301945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] debug = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.302530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.302530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.302530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.302530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.302530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.302530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.302530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_config_append = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_dir = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_file = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.303891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.304198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.304396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.304664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.305160 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.305344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] publish_errors = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.305577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.305810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.305990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.306235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.306574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.306816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] standard_threads = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.307047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.307290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.307467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.307642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_journal = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.307819 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_json = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.307993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_stderr = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.308166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] use_syslog = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.308397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.308652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.308888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.309145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.309382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.309631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.309874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.310120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.310355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.310627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.310874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.311123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.311355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.311602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.311939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.312129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.312376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.312616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.312869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.313121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.313367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.313600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.313851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.314108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.314360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.315464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.315687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.315897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.316131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.316368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.316620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.316869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.317100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.317329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.317576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.317821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.318058 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.318304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.318557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.318793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.319027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.319256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.319484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.319711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.319939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.320173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.320422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.320652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.320888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.321116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.321350 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.321584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.321829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.322059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.322295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.322553 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.322791 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.323027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.323263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.323491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.323739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.323971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.324215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.324445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.324714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.324947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.325177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.325448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.325648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.325916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.326152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.326401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.326666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.326902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.327138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.327373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.327618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.327856 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.328092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.328337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.328571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.328807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.329042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.329278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.329514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.329771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.330009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.330253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.330539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.330779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.331210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.331429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.331665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.331895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.332125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.332381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.332587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.332815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.333052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.333286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.333532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.333770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.334154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.334428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.334701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.334915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.external = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.335064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 12:00:51.335144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.335262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 12:00:51.336126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.336255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] command line args: [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 12:00:51.336360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.336480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 12:00:51.336480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ================================================================================ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 12:00:51.336717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.password = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.336717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] auth.token = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.336983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] admin_token = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.337178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.337287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] config_dir = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.337287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.337517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.337637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] config_source = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.337637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] debug = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.337883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.338019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.338166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.338287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.338392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.338545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.338650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.338779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.338897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.339020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.339141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.339254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.339367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.339479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_config_append = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.339479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.339687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.339801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_dir = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.339921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.340032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_file = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.340150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.340276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.340390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.340511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.340615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.340732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.340846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.340972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.341089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.341202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.341202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.341442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.341558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.341680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.341795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.341908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.342058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.342173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.342301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.342457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.342576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.342689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.342813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.342927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.343041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.343188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.343306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.343419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.343532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.343646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.343646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.343947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.344066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.344180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] publish_errors = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.344292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.344414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.344528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.344641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.344759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.344879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.345001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.345115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.345229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.345342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.345461 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.345576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] standard_threads = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.345696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.345811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.346459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.346581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.346581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.346818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.346941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_journal = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.347055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.347177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_json = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.347177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_stderr = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.347398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.347511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] use_syslog = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.347624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.347736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 12:00:51.347861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.347976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.348089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.348201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.348316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.348429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.348551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.348665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.348778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.348891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349276 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.349987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.350083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.350178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.350285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.350383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359024 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359903 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.359982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360420 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.360938 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.361064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.361140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.361211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.361280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.361358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.361435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.361610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.362805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.362914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.362992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.363965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.364910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.365970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.366947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.367986 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.368999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369069 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.369993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.370073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.370073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.370305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.370380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.370524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.370603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.370683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.370756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.370853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371099 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.371970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372407 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.372932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] role.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] role.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373291 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373569 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.373999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374275 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374438 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.external = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374881 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.374952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.password = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] auth.token = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.375990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.376926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.377004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.377077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.377164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.377791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.377888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.377962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.378922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.379941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380549 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.380997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.381073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.381145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.381223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.381377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.381461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.381656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.381740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.381914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.382004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.382157 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.382247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.382396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.382547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.382689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.382794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.382885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.382961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.383109 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.383189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.383347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.383426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.383584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.383665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.383820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.383899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.384065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.384143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.384299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.384379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.384532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.384610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.384777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.384855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.385026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.385105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.385260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.385345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.385504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.385583 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.385732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.385852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.385976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.386099 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.386214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.386330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.386490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 12:00:51.386610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.386726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.386965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.387203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.387443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.387693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.387938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.388193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.388428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.388678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.388929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.389168 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.389419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.389669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.389903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.390148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.390384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.390653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.390883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.391135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.391378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.391619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.391879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.392130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.392365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.392600 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.392882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.393691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.393921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.394199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.394460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.394699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.394948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.395212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.395450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.395714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.395966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.396217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.396527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.396688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.396928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.397166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.397409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.397643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.397894 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.398158 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.398482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.403667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.403667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.403667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.403667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.403667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.403667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.403667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.403667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.403667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.404969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.405707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.405707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.405707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.405707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.405960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] role.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.406141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] role.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.406377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.406658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.406914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.407167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.407419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.407669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.407991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.408247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.408656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.408956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.409215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.409474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.409730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.409984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.410442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.410705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.410999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.411203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.411457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.411714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.411959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.412223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.412481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.412725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.412961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.413198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.413434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.413689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.413947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.414200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.414462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.414706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.414941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.415183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.415422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.415675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.415917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.416178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.416430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.416671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.416918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.417167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.417417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.417912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.418175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.418427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.418688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.418941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.419175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.419423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.419737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.420004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.420254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.420568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.420750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.420994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.421233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.421474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.421725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.421963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.422306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.422481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.422750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.423246 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.423487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.423740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.423976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.424301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.424558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 12:00:51.424716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80603) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 12:00:51.426531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.426769 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.426996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.435855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.452330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.452563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.452792 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.462196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.465444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.465675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.483204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.483436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.519609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.522106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.571485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.571665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.572836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.573073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.646968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.647186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.647277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.647356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.647432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.647567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.647662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.647789 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.647873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.648002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.648084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.648217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.680021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.680244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.680467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.680685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.680902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.681109 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.681322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.681686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.681922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.682147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.682370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.682616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.682832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.683047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.693684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.694349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.707297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.707527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.707755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.707981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.708203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.708418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.708632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80603) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.709236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.709472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.709698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.709919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.710152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.710362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.710601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 12:00:51.731379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.731582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 12:00:51.731761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.731912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.731997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 12:00:51.732112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 12:00:51.732294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.732389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.732498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 12:00:51.732627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.732768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.732884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.733030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.733141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.733287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.733741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.744853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.745124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.745401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.745486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.745681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.745766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.745981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.746263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.754435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.755174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.757758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.758037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.758320 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.758618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.758740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.758899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.759025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.759183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.759279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.759469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.759550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.759756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.759953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.760049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.760127 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.760264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.760346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.760537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.760628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.760805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.761073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.761339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.787245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.788022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.788866 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.789144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.789656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.789935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.794775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.805393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.805393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.805607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.805607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.805607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.805939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.805939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.806080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.806080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.807950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.808234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.808513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.808731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.808815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.809019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.809101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.809303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.809388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.809569 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.809661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.809832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.809915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.810111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.810384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.810677 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.829329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.829521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.830104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.830969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.834639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.835526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.838236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.838558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.839159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.839456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.843489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.844414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.847724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.856929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.857039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.857810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.871893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.872276 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.872548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.872801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.872888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.873090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.873365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.873628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.881459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.881567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.883201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.883344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.883475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.883622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.886638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.886739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.886916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.887000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.887219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.887302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.887500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.887581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.887774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.887900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.901666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.901779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.901939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.905869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.905963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.906040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.906115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.906188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.906275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.906353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.906475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.906562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.906637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.906723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.922113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.922310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.925915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.926199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.929240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.929515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.930857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.931138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.932501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.932776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.932859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.933055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.934345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.934660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.941055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.942713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.942806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.944328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.945941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.946225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.948396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.948396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.948603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.948603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.948603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.948603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.956533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.956899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.957225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.958164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.958549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.958882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.965575 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.965871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.966152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.966457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.966734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.967004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.967087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80603) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 12:00:51.967296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.967379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.967562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.967645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.967838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.967918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.968105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.968270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.968380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.968460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.968639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.968720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.968916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.969005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.969175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.969256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.969485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.969893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.970154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.970462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:51.970731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80603) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 12:00:52.010525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d6dea7f9c0 pid: 80603 (default app) May 19 12:00:52.010721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d6dea7f9c0 pid: 80604 (default app) May 19 12:00:52.014191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae5ab0-9486-44b4-a7b5-344c99508726 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:00:52.014521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae5ab0-9486-44b4-a7b5-344c99508726 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:00:52.014808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae5ab0-9486-44b4-a7b5-344c99508726 None None] PATH_INFO: `/v3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:00:52.019136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu May 19 12:00:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:01.562011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036e80-3424-46a5-bf0a-2c5018b963d3 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:01.562580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036e80-3424-46a5-bf0a-2c5018b963d3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:01.562580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c036e80-3424-46a5-bf0a-2c5018b963d3 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:01.567819 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:01.574168 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:01.574696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:01.574880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:01.646029 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: INFO dbcounter [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] Registered counter for database keystone May 19 12:01:01.669130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG oslo_db.sqlalchemy.engines [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80603) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:01:01.677093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writer thread running {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:01:01.990905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG passlib.handlers.bcrypt [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80603) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 12:01:01.991334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG passlib.handlers.bcrypt [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80603) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 12:01:02.049961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:02.051693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:02.183326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 611 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 12:01:02.334584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c036e80-3424-46a5-bf0a-2c5018b963d3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:02.404478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: INFO dbcounter [None req-2c036e80-3424-46a5-bf0a-2c5018b963d3 None None] Registered counter for database keystone May 19 12:01:02.426336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG oslo_db.sqlalchemy.engines [None req-2c036e80-3424-46a5-bf0a-2c5018b963d3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 12:01:02.434097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writer thread running {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 12:01:02.563931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c036e80-3424-46a5-bf0a-2c5018b963d3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:02.569291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaca653-f7b8-4041-b11a-9071a6ea3d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:02.569960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaca653-f7b8-4041-b11a-9071a6ea3d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:02.570237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaca653-f7b8-4041-b11a-9071a6ea3d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:02.570540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaca653-f7b8-4041-b11a-9071a6ea3d4b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:02.579676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaca653-f7b8-4041-b11a-9071a6ea3d4b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:02.634058 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1eaca653-f7b8-4041-b11a-9071a6ea3d4b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 12:01:02.634397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaca653-f7b8-4041-b11a-9071a6ea3d4b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:02.645347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 12:01:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:01:02.651553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:02.666770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde0060-e129-44ae-9776-0b0db8b3a73e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:02.671617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc1b5a-826b-443b-afb9-b1ee82d87d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:02.672756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc1b5a-826b-443b-afb9-b1ee82d87d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:02.673034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc1b5a-826b-443b-afb9-b1ee82d87d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:02.673316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc1b5a-826b-443b-afb9-b1ee82d87d53 admin admin] PATH_INFO: `/v3/projects/67ffb05dd43a48979db84b48835b3e07` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:02.675484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc1b5a-826b-443b-afb9-b1ee82d87d53 admin admin] RBAC: Authorizing `identity:get_project(project_id=67ffb05dd43a48979db84b48835b3e07)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:02.728067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WARNING keystone.common.rbac_enforcer.enforcer [None req-58cc1b5a-826b-443b-afb9-b1ee82d87d53 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 12:01:02.728415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc1b5a-826b-443b-afb9-b1ee82d87d53 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:02.733323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu May 19 12:01:02 2022] GET /identity/v3/projects/67ffb05dd43a48979db84b48835b3e07 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:05.016902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213857e-aa7e-4a2c-9201-2d9547bde79e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:05.017443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213857e-aa7e-4a2c-9201-2d9547bde79e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:05.018134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213857e-aa7e-4a2c-9201-2d9547bde79e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:05.022666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:05.028309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fa1ec-9887-435b-bbcf-fe905cc7830d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:05.028755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fa1ec-9887-435b-bbcf-fe905cc7830d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:05.028879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fa1ec-9887-435b-bbcf-fe905cc7830d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:05.048336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8b6fa1ec-9887-435b-bbcf-fe905cc7830d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:05.050762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8b6fa1ec-9887-435b-bbcf-fe905cc7830d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:05.058605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 12:01:05.081658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213857e-aa7e-4a2c-9201-2d9547bde79e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:05.093761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213857e-aa7e-4a2c-9201-2d9547bde79e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:05.098618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367d8cb-72c2-4085-99bc-375a2feaa9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:05.099248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367d8cb-72c2-4085-99bc-375a2feaa9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:05.099533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367d8cb-72c2-4085-99bc-375a2feaa9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:05.099807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367d8cb-72c2-4085-99bc-375a2feaa9f8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:05.107880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367d8cb-72c2-4085-99bc-375a2feaa9f8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:05.109660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367d8cb-72c2-4085-99bc-375a2feaa9f8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:05.119386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 19 12:01:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:01:05.125059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fa1ec-9887-435b-bbcf-fe905cc7830d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:05.134343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fa1ec-9887-435b-bbcf-fe905cc7830d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:05.138741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165402f-7bfc-4c50-bb61-0aba10af8b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:05.139606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165402f-7bfc-4c50-bb61-0aba10af8b85 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:05.139891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165402f-7bfc-4c50-bb61-0aba10af8b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:05.140168 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165402f-7bfc-4c50-bb61-0aba10af8b85 admin admin] PATH_INFO: `/v3/users/87e06b5266a1425fb48c02407f951f73` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:05.141558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165402f-7bfc-4c50-bb61-0aba10af8b85 admin admin] RBAC: Authorizing `identity:get_user(user_id=87e06b5266a1425fb48c02407f951f73)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:05.143584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165402f-7bfc-4c50-bb61-0aba10af8b85 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:05.145198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu May 19 12:01:05 2022] GET /identity/v3/users/87e06b5266a1425fb48c02407f951f73 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:07.475737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee632d5-e6b2-402a-af73-c31992c0d30a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:07.476292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee632d5-e6b2-402a-af73-c31992c0d30a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:07.476292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee632d5-e6b2-402a-af73-c31992c0d30a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:07.477621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:07.482946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6ee6-0470-4752-9655-6e17500dc50b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:07.483351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6ee6-0470-4752-9655-6e17500dc50b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:07.483521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6ee6-0470-4752-9655-6e17500dc50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:07.500589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-90fb6ee6-0470-4752-9655-6e17500dc50b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:07.502380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-90fb6ee6-0470-4752-9655-6e17500dc50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:07.512521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a866549-817c-4768-85bb-8ace934b370e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:07.512813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a866549-817c-4768-85bb-8ace934b370e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:07.513093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a866549-817c-4768-85bb-8ace934b370e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:07.514463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:07.519601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a28b6-9205-4ac1-8a0d-135d2df111ae None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:07.519892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a28b6-9205-4ac1-8a0d-135d2df111ae None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:07.520176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a28b6-9205-4ac1-8a0d-135d2df111ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:07.563033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG passlib.handlers.bcrypt [None req-e91a28b6-9205-4ac1-8a0d-135d2df111ae None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80604) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 12:01:07.563464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG passlib.handlers.bcrypt [None req-e91a28b6-9205-4ac1-8a0d-135d2df111ae None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80604) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 12:01:07.586654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:07.608917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb6ee6-0470-4752-9655-6e17500dc50b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:07.611640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e91a28b6-9205-4ac1-8a0d-135d2df111ae None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:07.613399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e91a28b6-9205-4ac1-8a0d-135d2df111ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:07.634174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:07.713681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:07.720498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb6ee6-0470-4752-9655-6e17500dc50b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:07.735391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a28b6-9205-4ac1-8a0d-135d2df111ae None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:07.755449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad3c61-14f8-411a-a6da-2e6875e20559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:07.756292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad3c61-14f8-411a-a6da-2e6875e20559 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:07.756579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad3c61-14f8-411a-a6da-2e6875e20559 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:07.756858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad3c61-14f8-411a-a6da-2e6875e20559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:07.770045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad3c61-14f8-411a-a6da-2e6875e20559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d4ad3c61-14f8-411a-a6da-2e6875e20559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:07.781731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:07.782806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:07.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:01:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:07.789551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad3c61-14f8-411a-a6da-2e6875e20559 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:07.881254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:07.899666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad3c61-14f8-411a-a6da-2e6875e20559 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:07.934457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03151824-11b4-41d6-86f5-baff457330e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:07.935131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151824-11b4-41d6-86f5-baff457330e1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:07.935413 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151824-11b4-41d6-86f5-baff457330e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:07.935689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151824-11b4-41d6-86f5-baff457330e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:07.936436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03151824-11b4-41d6-86f5-baff457330e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:07.936748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03151824-11b4-41d6-86f5-baff457330e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:07.938137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03151824-11b4-41d6-86f5-baff457330e1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:07.950461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:07.951338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a28b6-9205-4ac1-8a0d-135d2df111ae None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:07.953780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:07.955964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03151824-11b4-41d6-86f5-baff457330e1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:07.991042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6f8d-5ba3-46be-8f56-b78947b80913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:07.991919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6f8d-5ba3-46be-8f56-b78947b80913 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:07.992209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6f8d-5ba3-46be-8f56-b78947b80913 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:07.992487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6f8d-5ba3-46be-8f56-b78947b80913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:08.057398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03151824-11b4-41d6-86f5-baff457330e1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:08.063403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6f8d-5ba3-46be-8f56-b78947b80913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:08.064972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6f8d-5ba3-46be-8f56-b78947b80913 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:08.090846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e2c39-9b8b-45ec-8548-65f328612713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:08.091726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e2c39-9b8b-45ec-8548-65f328612713 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:08.092017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e2c39-9b8b-45ec-8548-65f328612713 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:08.092297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e2c39-9b8b-45ec-8548-65f328612713 None admin] PATH_INFO: `/v3/users/87e06b5266a1425fb48c02407f951f73` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:08.093513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e2c39-9b8b-45ec-8548-65f328612713 None admin] RBAC: Authorizing `identity:get_user(user_id=87e06b5266a1425fb48c02407f951f73)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:08.095547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e2c39-9b8b-45ec-8548-65f328612713 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:08.097210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 9/16] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:07 2022] GET /identity/v3/users/87e06b5266a1425fb48c02407f951f73 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:08.102614 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e2c39-9b8b-45ec-8548-65f328612713 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:08.117377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:08.166436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 8/17] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 12:01:08.172209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6f8d-5ba3-46be-8f56-b78947b80913 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:08.176086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:08.210552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e2c39-9b8b-45ec-8548-65f328612713 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:08.246002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33a204-91f2-42b6-9713-4467ff7ca694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:08.246733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33a204-91f2-42b6-9713-4467ff7ca694 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:08.247024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33a204-91f2-42b6-9713-4467ff7ca694 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:08.247305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33a204-91f2-42b6-9713-4467ff7ca694 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:08.248506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33a204-91f2-42b6-9713-4467ff7ca694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:08.249955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33a204-91f2-42b6-9713-4467ff7ca694 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:08.252161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 10/18] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:08.258034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33a204-91f2-42b6-9713-4467ff7ca694 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:08.275485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6f8d-5ba3-46be-8f56-b78947b80913 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:08.312032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e47a79-3398-42ac-a385-5c57674f192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:08.312824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47a79-3398-42ac-a385-5c57674f192b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:08.313146 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47a79-3398-42ac-a385-5c57674f192b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:08.313428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e47a79-3398-42ac-a385-5c57674f192b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:08.334570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e47a79-3398-42ac-a385-5c57674f192b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:08.336178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e47a79-3398-42ac-a385-5c57674f192b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:08.341083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-94e47a79-3398-42ac-a385-5c57674f192b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:08.358601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:08.359654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:08.360444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:08.360444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:08.360444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:08.360444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:08.360444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 9/19] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:08.360736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33a204-91f2-42b6-9713-4467ff7ca694 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:08.366396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e47a79-3398-42ac-a385-5c57674f192b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:08.394868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5c6f1-cb84-4e08-bad2-836e089e70a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:08.395625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5c6f1-cb84-4e08-bad2-836e089e70a7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:08.395978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5c6f1-cb84-4e08-bad2-836e089e70a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:08.396269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5c6f1-cb84-4e08-bad2-836e089e70a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:08.397110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5c6f1-cb84-4e08-bad2-836e089e70a7 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.domain.id=default, user.id=87e06b5266a1425fb48c02407f951f73) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:08.397422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5c6f1-cb84-4e08-bad2-836e089e70a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:08.398911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5c6f1-cb84-4e08-bad2-836e089e70a7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:08.406197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 12:01:08 2022] GET /identity/v3/role_assignments?user.id=87e06b5266a1425fb48c02407f951f73&scope.domain.id=default&role.id=bd295cb5d1194b469beff0381c6b23af => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:08.429594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:08.478161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e47a79-3398-42ac-a385-5c57674f192b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:08.516461 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92368f65-8191-4c13-9498-ebeba2c8550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:08.517078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92368f65-8191-4c13-9498-ebeba2c8550a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:08.517366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92368f65-8191-4c13-9498-ebeba2c8550a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:08.517706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92368f65-8191-4c13-9498-ebeba2c8550a None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:08.518391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92368f65-8191-4c13-9498-ebeba2c8550a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:08.518732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92368f65-8191-4c13-9498-ebeba2c8550a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:08.520183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92368f65-8191-4c13-9498-ebeba2c8550a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:08.539673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:11.232095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a980c8-f5bc-4f70-853f-68b29dd0dd04 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:11.232617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a980c8-f5bc-4f70-853f-68b29dd0dd04 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:11.232721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a980c8-f5bc-4f70-853f-68b29dd0dd04 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:11.234071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:11.258111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d01d50-e5ca-494d-bb54-c7320c92c71e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:11.259648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d01d50-e5ca-494d-bb54-c7320c92c71e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:11.260433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d01d50-e5ca-494d-bb54-c7320c92c71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:11.299591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-12d01d50-e5ca-494d-bb54-c7320c92c71e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:11.302038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-12d01d50-e5ca-494d-bb54-c7320c92c71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:11.474442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 11/23] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:11.510849 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a980c8-f5bc-4f70-853f-68b29dd0dd04 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:11.763199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a980c8-f5bc-4f70-853f-68b29dd0dd04 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:11.809944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4dcd2-031a-45c6-aca6-5cc216993df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:11.810783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4dcd2-031a-45c6-aca6-5cc216993df9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:11.811073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4dcd2-031a-45c6-aca6-5cc216993df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:11.811358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4dcd2-031a-45c6-aca6-5cc216993df9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:11.831515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4dcd2-031a-45c6-aca6-5cc216993df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b6d4dcd2-031a-45c6-aca6-5cc216993df9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:11.839476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:11.840930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:11.842270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 13/24] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:01:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:11.848722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d01d50-e5ca-494d-bb54-c7320c92c71e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:11.865324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4b50-60fc-429d-ad7e-70b9453a18f0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:11.865778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4b50-60fc-429d-ad7e-70b9453a18f0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:11.866018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4b50-60fc-429d-ad7e-70b9453a18f0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:11.868254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 14/25] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:11.875076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8aeeb-1042-40ae-8305-5aa3d1bedf5f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:11.875784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8aeeb-1042-40ae-8305-5aa3d1bedf5f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:11.875784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8aeeb-1042-40ae-8305-5aa3d1bedf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:11.892934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:11.913570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-05f8aeeb-1042-40ae-8305-5aa3d1bedf5f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:11.914885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-05f8aeeb-1042-40ae-8305-5aa3d1bedf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:12.049369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:12.128470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 15/26] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:12.164859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8aeeb-1042-40ae-8305-5aa3d1bedf5f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:12.251953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d01d50-e5ca-494d-bb54-c7320c92c71e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:12.324226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a038d98-d419-4205-8889-ed443625442c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:12.326433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a038d98-d419-4205-8889-ed443625442c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:12.327148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a038d98-d419-4205-8889-ed443625442c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:12.327850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a038d98-d419-4205-8889-ed443625442c None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:12.329555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a038d98-d419-4205-8889-ed443625442c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:12.330339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a038d98-d419-4205-8889-ed443625442c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:12.336077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a038d98-d419-4205-8889-ed443625442c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:12.359940 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 12/27] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:12.362765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255b659-fc43-464e-b1c2-cf4a2c282425 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:12.363498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255b659-fc43-464e-b1c2-cf4a2c282425 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:12.364179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255b659-fc43-464e-b1c2-cf4a2c282425 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:12.367776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 13/28] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:12.369497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255b659-fc43-464e-b1c2-cf4a2c282425 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:12.452163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8aeeb-1042-40ae-8305-5aa3d1bedf5f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:12.508903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc157c-feae-4811-a486-20b8b64cce2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:12.510092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc157c-feae-4811-a486-20b8b64cce2e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:12.510634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc157c-feae-4811-a486-20b8b64cce2e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:12.511139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc157c-feae-4811-a486-20b8b64cce2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:12.514583 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:12.523648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc157c-feae-4811-a486-20b8b64cce2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:12.525646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc157c-feae-4811-a486-20b8b64cce2e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:12.553224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255b659-fc43-464e-b1c2-cf4a2c282425 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-44bc157c-feae-4811-a486-20b8b64cce2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:01:12.562616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:12.565052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:12.567076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:01:12.567076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:01:12.567076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:01:12.567076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:12.567977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:12.575085 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 16/29] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:12.577885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc731738-f60e-4527-a360-c1e1980282f1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:12.578706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc731738-f60e-4527-a360-c1e1980282f1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:12.580004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc731738-f60e-4527-a360-c1e1980282f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:12.599702 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca51f44-f15d-4a5a-b228-894baa30bbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:12.600800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca51f44-f15d-4a5a-b228-894baa30bbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:12.601190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca51f44-f15d-4a5a-b228-894baa30bbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:12.601570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca51f44-f15d-4a5a-b228-894baa30bbf1 None admin] PATH_INFO: `/v3/users/87e06b5266a1425fb48c02407f951f73` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:12.603341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca51f44-f15d-4a5a-b228-894baa30bbf1 None admin] RBAC: Authorizing `identity:get_user(user_id=87e06b5266a1425fb48c02407f951f73)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:12.604978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca51f44-f15d-4a5a-b228-894baa30bbf1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:12.606929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 14/30] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:12 2022] GET /identity/v3/users/87e06b5266a1425fb48c02407f951f73 => generated 247 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:12.609329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800bfcf-45df-419f-a113-a42df2ae2f54 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:12.610125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800bfcf-45df-419f-a113-a42df2ae2f54 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:12.611437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800bfcf-45df-419f-a113-a42df2ae2f54 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:12.616183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 15/31] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:12.617189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800bfcf-45df-419f-a113-a42df2ae2f54 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:12.624383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-dc731738-f60e-4527-a360-c1e1980282f1 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:12.626975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dc731738-f60e-4527-a360-c1e1980282f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:12.744572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800bfcf-45df-419f-a113-a42df2ae2f54 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:12.752536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 17/32] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:12.753640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc731738-f60e-4527-a360-c1e1980282f1 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:12.786784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c9ac6-8f22-4f1d-a77d-c643911c8907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:12.788015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c9ac6-8f22-4f1d-a77d-c643911c8907 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:12.788423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c9ac6-8f22-4f1d-a77d-c643911c8907 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:12.788810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c9ac6-8f22-4f1d-a77d-c643911c8907 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:12.789964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c9ac6-8f22-4f1d-a77d-c643911c8907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:12.790382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c9ac6-8f22-4f1d-a77d-c643911c8907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:12.793147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7c9ac6-8f22-4f1d-a77d-c643911c8907 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:12.808332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 16/33] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:12.813622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e9c05-c10b-4a8d-a0c8-0b9a87fb9be1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:12.813622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e9c05-c10b-4a8d-a0c8-0b9a87fb9be1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:12.813622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e9c05-c10b-4a8d-a0c8-0b9a87fb9be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:12.831857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-323e9c05-c10b-4a8d-a0c8-0b9a87fb9be1 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:12.832843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-323e9c05-c10b-4a8d-a0c8-0b9a87fb9be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:12.868007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:12.882524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc731738-f60e-4527-a360-c1e1980282f1 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:12.902491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:12.916210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 17/34] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:12.924255 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e9c05-c10b-4a8d-a0c8-0b9a87fb9be1 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:12.930752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4822a-0247-436e-9be6-ddb5122631d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:12.931832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4822a-0247-436e-9be6-ddb5122631d7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:12.933199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4822a-0247-436e-9be6-ddb5122631d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:12.933633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4822a-0247-436e-9be6-ddb5122631d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:12.939621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4822a-0247-436e-9be6-ddb5122631d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:12.941727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4822a-0247-436e-9be6-ddb5122631d7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:12.945231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 18/35] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:12.946083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4822a-0247-436e-9be6-ddb5122631d7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:13.085456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e9c05-c10b-4a8d-a0c8-0b9a87fb9be1 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:13.094157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4822a-0247-436e-9be6-ddb5122631d7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:13.126867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb3046-384e-4326-a1f2-9ab23c28ea4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:13.128673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb3046-384e-4326-a1f2-9ab23c28ea4f None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:13.129199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb3046-384e-4326-a1f2-9ab23c28ea4f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:13.130081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb3046-384e-4326-a1f2-9ab23c28ea4f None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:13.131943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb3046-384e-4326-a1f2-9ab23c28ea4f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:13.136616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb3046-384e-4326-a1f2-9ab23c28ea4f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:13.141712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf3f24-3b5b-4749-a9d4-82ee059b3294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:13.143114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf3f24-3b5b-4749-a9d4-82ee059b3294 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:13.143659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf3f24-3b5b-4749-a9d4-82ee059b3294 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:13.144138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf3f24-3b5b-4749-a9d4-82ee059b3294 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:13.145452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf3f24-3b5b-4749-a9d4-82ee059b3294 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:13.147610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf3f24-3b5b-4749-a9d4-82ee059b3294 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:13.427082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 19/36] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 12:01:12 2022] POST /identity/v3/roles => generated 225 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:13.428747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf3f24-3b5b-4749-a9d4-82ee059b3294 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:13.468660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 18/37] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:01:12 2022] POST /identity/v3/projects => generated 301 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:13.468660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb3046-384e-4326-a1f2-9ab23c28ea4f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:13.517792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:13.550807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:13.668898 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf3f24-3b5b-4749-a9d4-82ee059b3294 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:13.747044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb3046-384e-4326-a1f2-9ab23c28ea4f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:13.778823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5d655-3001-4e5b-9a90-e4887bf3cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:13.780101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d655-3001-4e5b-9a90-e4887bf3cd13 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:13.780617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d655-3001-4e5b-9a90-e4887bf3cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:13.781131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d655-3001-4e5b-9a90-e4887bf3cd13 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:13.782170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5d655-3001-4e5b-9a90-e4887bf3cd13 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:13.787852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5d655-3001-4e5b-9a90-e4887bf3cd13 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:13.827652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bede91-5e1f-48de-a1b4-0814620c5594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:13.828996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bede91-5e1f-48de-a1b4-0814620c5594 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:13.829511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bede91-5e1f-48de-a1b4-0814620c5594 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:13.830004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bede91-5e1f-48de-a1b4-0814620c5594 None admin] PATH_INFO: `/v3/domains/default/users/87e06b5266a1425fb48c02407f951f73/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:13.836289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bede91-5e1f-48de-a1b4-0814620c5594 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=87e06b5266a1425fb48c02407f951f73, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:13.838966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bede91-5e1f-48de-a1b4-0814620c5594 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:13.889154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 20/38] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 12:01:13 2022] POST /identity/v3/roles => generated 219 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:13.892192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe14d46-6d8b-4f8b-b04c-7b324c2017bc None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:13.892688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe14d46-6d8b-4f8b-b04c-7b324c2017bc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:13.893106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe14d46-6d8b-4f8b-b04c-7b324c2017bc None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:13.895364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 21/39] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:13.897347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d7f23-ca22-4fd6-8c8b-48263b668e10 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:13.897789 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d7f23-ca22-4fd6-8c8b-48263b668e10 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:13.898202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d7f23-ca22-4fd6-8c8b-48263b668e10 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:13.900061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 22/40] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:13.902471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddc27c-a893-4d75-b315-36a4081aaa78 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:13.902999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddc27c-a893-4d75-b315-36a4081aaa78 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:13.903507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddc27c-a893-4d75-b315-36a4081aaa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:13.933830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c4ddc27c-a893-4d75-b315-36a4081aaa78 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:13.936434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c4ddc27c-a893-4d75-b315-36a4081aaa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:14.017952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 19/41] 10.222.0.57 () {58 vars in 1361 bytes} [Thu May 19 12:01:13 2022] PUT /identity/v3/domains/default/users/87e06b5266a1425fb48c02407f951f73/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:14.021681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75205f5-df1b-46d0-96c8-02d5a947f407 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:14.021681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75205f5-df1b-46d0-96c8-02d5a947f407 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:14.023196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75205f5-df1b-46d0-96c8-02d5a947f407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:14.076273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d75205f5-df1b-46d0-96c8-02d5a947f407 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:14.081766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d75205f5-df1b-46d0-96c8-02d5a947f407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:14.090057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 23/42] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:14.128616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddc27c-a893-4d75-b315-36a4081aaa78 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:14.182864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:14.198063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 20/43] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:14.226763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75205f5-df1b-46d0-96c8-02d5a947f407 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:14.244524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:14.287128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddc27c-a893-4d75-b315-36a4081aaa78 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:14.325848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf669a2-ed6b-461a-bdc4-0386162b3354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:14.326841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf669a2-ed6b-461a-bdc4-0386162b3354 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:14.327127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf669a2-ed6b-461a-bdc4-0386162b3354 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:14.327455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf669a2-ed6b-461a-bdc4-0386162b3354 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:14.328872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf669a2-ed6b-461a-bdc4-0386162b3354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:14.331159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf669a2-ed6b-461a-bdc4-0386162b3354 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:14.333076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 24/44] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:14.338528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf669a2-ed6b-461a-bdc4-0386162b3354 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:14.361169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75205f5-df1b-46d0-96c8-02d5a947f407 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:14.401792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5077fd-d40c-49a2-81ef-501eb5297eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:14.402401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5077fd-d40c-49a2-81ef-501eb5297eef None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:14.402833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5077fd-d40c-49a2-81ef-501eb5297eef None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:14.403023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5077fd-d40c-49a2-81ef-501eb5297eef None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:14.403779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5077fd-d40c-49a2-81ef-501eb5297eef None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:14.405263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5077fd-d40c-49a2-81ef-501eb5297eef None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:14.450241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf669a2-ed6b-461a-bdc4-0386162b3354 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:14.452003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:14.487081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83339cd6-d029-43b9-8939-5dbd5da47645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:14.487740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83339cd6-d029-43b9-8939-5dbd5da47645 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:14.488032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83339cd6-d029-43b9-8939-5dbd5da47645 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:14.488333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83339cd6-d029-43b9-8939-5dbd5da47645 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:14.489049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83339cd6-d029-43b9-8939-5dbd5da47645 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:14.490511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83339cd6-d029-43b9-8939-5dbd5da47645 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:14.544026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 21/45] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 12:01:14 2022] POST /identity/v3/roles => generated 223 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:14.671105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 25/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:01:14 2022] POST /identity/v3/projects => generated 312 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:01:16.376725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbb422-dafd-45b2-b3b3-647b106ee93e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:16.377508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbb422-dafd-45b2-b3b3-647b106ee93e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:16.377508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbb422-dafd-45b2-b3b3-647b106ee93e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:16.378740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 22/47] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:16.384097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee408872-e356-458c-9a17-d39a9272cf25 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:16.384581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee408872-e356-458c-9a17-d39a9272cf25 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:16.384672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee408872-e356-458c-9a17-d39a9272cf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:16.402080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ee408872-e356-458c-9a17-d39a9272cf25 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:16.403294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ee408872-e356-458c-9a17-d39a9272cf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:16.472966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 26/48] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:16.497070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbb422-dafd-45b2-b3b3-647b106ee93e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:16.568890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:16.603862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbb422-dafd-45b2-b3b3-647b106ee93e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:16.638989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701351f-c6d7-4846-b1e2-c93c47d523a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:16.639707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701351f-c6d7-4846-b1e2-c93c47d523a1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:16.639994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701351f-c6d7-4846-b1e2-c93c47d523a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:16.640275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701351f-c6d7-4846-b1e2-c93c47d523a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:16.652786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701351f-c6d7-4846-b1e2-c93c47d523a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1701351f-c6d7-4846-b1e2-c93c47d523a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:16.662521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:16.663646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:16.664629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 23/49] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:01:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:16.669859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee408872-e356-458c-9a17-d39a9272cf25 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:16.753864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:16.772647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee408872-e356-458c-9a17-d39a9272cf25 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:16.815262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222a793-1ec2-448b-bfad-edbb306b1706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:16.815954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222a793-1ec2-448b-bfad-edbb306b1706 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:16.816245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222a793-1ec2-448b-bfad-edbb306b1706 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:16.816526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222a793-1ec2-448b-bfad-edbb306b1706 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:16.817297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222a793-1ec2-448b-bfad-edbb306b1706 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:16.817609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222a793-1ec2-448b-bfad-edbb306b1706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:16.819073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222a793-1ec2-448b-bfad-edbb306b1706 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:16.828793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 27/50] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:16.835407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701351f-c6d7-4846-b1e2-c93c47d523a1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:16.943233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701351f-c6d7-4846-b1e2-c93c47d523a1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:16.966503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08695cfd-5975-4407-9bca-d2a0f80f9b7f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:16.966796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08695cfd-5975-4407-9bca-d2a0f80f9b7f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:16.967079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08695cfd-5975-4407-9bca-d2a0f80f9b7f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:16.968472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 28/51] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:16.973049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:16.974462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35d2d4-c1c8-41a2-aea1-63258d58e0a9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:16.974757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35d2d4-c1c8-41a2-aea1-63258d58e0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:16.975040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35d2d4-c1c8-41a2-aea1-63258d58e0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:16.980639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce9dc-37a4-447a-a1c4-62a4e3ff8738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:16.981568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce9dc-37a4-447a-a1c4-62a4e3ff8738 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:16.981858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce9dc-37a4-447a-a1c4-62a4e3ff8738 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:16.982142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce9dc-37a4-447a-a1c4-62a4e3ff8738 None admin] PATH_INFO: `/v3/users/87e06b5266a1425fb48c02407f951f73` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:16.983399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fce9dc-37a4-447a-a1c4-62a4e3ff8738 None admin] RBAC: Authorizing `identity:get_user(user_id=87e06b5266a1425fb48c02407f951f73)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:16.991464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fce9dc-37a4-447a-a1c4-62a4e3ff8738 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:16.991464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 24/52] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:16 2022] GET /identity/v3/users/87e06b5266a1425fb48c02407f951f73 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:16.992538 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce9dc-37a4-447a-a1c4-62a4e3ff8738 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:16.992807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7a35d2d4-c1c8-41a2-aea1-63258d58e0a9 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:16.994024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7a35d2d4-c1c8-41a2-aea1-63258d58e0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:17.067773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 29/53] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:17.092063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35d2d4-c1c8-41a2-aea1-63258d58e0a9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:17.107803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce9dc-37a4-447a-a1c4-62a4e3ff8738 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:17.149608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c11f69-fa43-4efa-91c5-e0d5e167ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:17.150432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c11f69-fa43-4efa-91c5-e0d5e167ca6d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:17.150732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c11f69-fa43-4efa-91c5-e0d5e167ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:17.151011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c11f69-fa43-4efa-91c5-e0d5e167ca6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:17.152218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c11f69-fa43-4efa-91c5-e0d5e167ca6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:17.153678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c11f69-fa43-4efa-91c5-e0d5e167ca6d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:17.155315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 25/54] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:17.161555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c11f69-fa43-4efa-91c5-e0d5e167ca6d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:17.174902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:17.202230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35d2d4-c1c8-41a2-aea1-63258d58e0a9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:17.230097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:17.241075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61de139-8b57-4217-bb2b-e32e850e9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:17.245300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61de139-8b57-4217-bb2b-e32e850e9537 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:17.245300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61de139-8b57-4217-bb2b-e32e850e9537 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:17.245300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61de139-8b57-4217-bb2b-e32e850e9537 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:17.245300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61de139-8b57-4217-bb2b-e32e850e9537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:17.245300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61de139-8b57-4217-bb2b-e32e850e9537 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:17.246857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 30/55] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:17.252491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61de139-8b57-4217-bb2b-e32e850e9537 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:17.268838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c11f69-fa43-4efa-91c5-e0d5e167ca6d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:17.316424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68fa78-132f-46a3-a302-24a483b972d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:17.317093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68fa78-132f-46a3-a302-24a483b972d6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:17.317506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68fa78-132f-46a3-a302-24a483b972d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:17.318027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68fa78-132f-46a3-a302-24a483b972d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:17.319017 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68fa78-132f-46a3-a302-24a483b972d6 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.domain.id=default, user.id=87e06b5266a1425fb48c02407f951f73) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:17.319339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68fa78-132f-46a3-a302-24a483b972d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:17.320837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68fa78-132f-46a3-a302-24a483b972d6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:17.336450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 26/56] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 19 12:01:17 2022] GET /identity/v3/role_assignments?user.id=87e06b5266a1425fb48c02407f951f73&scope.domain.id=default&role.id=bd295cb5d1194b469beff0381c6b23af => generated 530 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:17.361804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61de139-8b57-4217-bb2b-e32e850e9537 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:17.400697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764b72-33ad-472a-bb86-8d254ea661fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:17.401395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764b72-33ad-472a-bb86-8d254ea661fa None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:17.401652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764b72-33ad-472a-bb86-8d254ea661fa None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:17.401926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764b72-33ad-472a-bb86-8d254ea661fa None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:17.402756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69764b72-33ad-472a-bb86-8d254ea661fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:17.404225 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69764b72-33ad-472a-bb86-8d254ea661fa None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:17.435107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:17.573533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 31/57] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 12:01:17 2022] POST /identity/v3/projects => generated 298 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:01:19.827217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7edd2d-ae53-4340-9217-f5af399f073e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:19.827737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7edd2d-ae53-4340-9217-f5af399f073e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:19.827863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7edd2d-ae53-4340-9217-f5af399f073e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:19.829234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 27/58] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:19.834676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3295c7-04eb-47a0-96d2-605357d677a3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:19.835088 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3295c7-04eb-47a0-96d2-605357d677a3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:19.835282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3295c7-04eb-47a0-96d2-605357d677a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:19.852242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2b3295c7-04eb-47a0-96d2-605357d677a3 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:19.853419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2b3295c7-04eb-47a0-96d2-605357d677a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:19.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 32/59] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:19.948929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7edd2d-ae53-4340-9217-f5af399f073e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:20.051729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7edd2d-ae53-4340-9217-f5af399f073e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:20.076600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:20.092308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a0ca7-96b7-4c6a-8844-4ed3fc10552d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:20.092308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a0ca7-96b7-4c6a-8844-4ed3fc10552d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:20.092308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a0ca7-96b7-4c6a-8844-4ed3fc10552d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:20.092308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a0ca7-96b7-4c6a-8844-4ed3fc10552d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:20.092737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a0ca7-96b7-4c6a-8844-4ed3fc10552d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:20.092737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a0ca7-96b7-4c6a-8844-4ed3fc10552d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:20.092737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 28/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:20.096297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3295c7-04eb-47a0-96d2-605357d677a3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:20.197966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3295c7-04eb-47a0-96d2-605357d677a3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:20.225431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:20.234072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb23e2-3199-40aa-b9e7-c7ab586be7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:20.240989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb23e2-3199-40aa-b9e7-c7ab586be7bd None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:20.240989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb23e2-3199-40aa-b9e7-c7ab586be7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:20.240989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb23e2-3199-40aa-b9e7-c7ab586be7bd None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:20.240989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb23e2-3199-40aa-b9e7-c7ab586be7bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:20.240989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb23e2-3199-40aa-b9e7-c7ab586be7bd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:20.364661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 33/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:01:20 2022] POST /identity/v3/users => generated 275 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:01:23.929412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426d25-f4d7-4a3b-997b-d320f19a8a20 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:23.930526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426d25-f4d7-4a3b-997b-d320f19a8a20 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:23.931033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23426d25-f4d7-4a3b-997b-d320f19a8a20 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:23.932777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 29/62] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:23.939315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d3635-8386-486d-a6b3-9b96693d4682 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:23.940304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d3635-8386-486d-a6b3-9b96693d4682 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:23.940803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d3635-8386-486d-a6b3-9b96693d4682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:23.971691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2d1d3635-8386-486d-a6b3-9b96693d4682 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:23.972969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2d1d3635-8386-486d-a6b3-9b96693d4682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:24.076162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 34/63] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:24.124033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23426d25-f4d7-4a3b-997b-d320f19a8a20 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:24.135490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06cce1-aec1-4a66-9b8a-e7f31a6d4988 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.136331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06cce1-aec1-4a66-9b8a-e7f31a6d4988 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.139159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06cce1-aec1-4a66-9b8a-e7f31a6d4988 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.141534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 35/64] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:24 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:24.151997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50706ad5-34a0-43fd-9190-2a37dde74a8f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.152650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50706ad5-34a0-43fd-9190-2a37dde74a8f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.153150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50706ad5-34a0-43fd-9190-2a37dde74a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.192595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-50706ad5-34a0-43fd-9190-2a37dde74a8f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:24.194240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-50706ad5-34a0-43fd-9190-2a37dde74a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:24.334113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23426d25-f4d7-4a3b-997b-d320f19a8a20 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:24.335666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 36/65] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:24.337134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116f32f-7ddc-4a0c-b942-7a1f402912a5 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.337430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116f32f-7ddc-4a0c-b942-7a1f402912a5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.337883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116f32f-7ddc-4a0c-b942-7a1f402912a5 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.339380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 37/66] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:24.351929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef841a-3a0f-49b6-be15-2a0795084e24 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.352471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef841a-3a0f-49b6-be15-2a0795084e24 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.352908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef841a-3a0f-49b6-be15-2a0795084e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.372263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-abef841a-3a0f-49b6-be15-2a0795084e24 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:24.374019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-abef841a-3a0f-49b6-be15-2a0795084e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:24.374261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0531a-7ffa-4adb-b9ee-dc7b48d67992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:24.375877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0531a-7ffa-4adb-b9ee-dc7b48d67992 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.376425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0531a-7ffa-4adb-b9ee-dc7b48d67992 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.376924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0531a-7ffa-4adb-b9ee-dc7b48d67992 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.391668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0531a-7ffa-4adb-b9ee-dc7b48d67992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-58e0531a-7ffa-4adb-b9ee-dc7b48d67992 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:24.401943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:24.402863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:24.403614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:24.405963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 30/67] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:24.407564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0531a-7ffa-4adb-b9ee-dc7b48d67992 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:24.423602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:24.466644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 38/68] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:24.467956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef841a-3a0f-49b6-be15-2a0795084e24 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:24.475775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:24.526532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0531a-7ffa-4adb-b9ee-dc7b48d67992 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:24.572589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc97cb5-134d-4514-833b-8a5c35d89bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:24.573713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc97cb5-134d-4514-833b-8a5c35d89bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.574215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc97cb5-134d-4514-833b-8a5c35d89bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.574527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc97cb5-134d-4514-833b-8a5c35d89bd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.587391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc97cb5-134d-4514-833b-8a5c35d89bd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:24.588217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef841a-3a0f-49b6-be15-2a0795084e24 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0cc97cb5-134d-4514-833b-8a5c35d89bd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:24.595802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:24.596817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:24.597659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 31/69] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:01:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:24.598357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a59d2-6946-419a-9a5d-a53e5fea24fd None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.598601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a59d2-6946-419a-9a5d-a53e5fea24fd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.599029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a59d2-6946-419a-9a5d-a53e5fea24fd None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.600464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 32/70] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:24.605277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a59d2-6946-419a-9a5d-a53e5fea24fd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:24.640537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c8a54-6947-4872-a3a2-bbf6786649e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:24.642062 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c8a54-6947-4872-a3a2-bbf6786649e3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.642799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c8a54-6947-4872-a3a2-bbf6786649e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.643486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c8a54-6947-4872-a3a2-bbf6786649e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.644927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c8a54-6947-4872-a3a2-bbf6786649e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:24.645595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c8a54-6947-4872-a3a2-bbf6786649e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:24.647923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c8a54-6947-4872-a3a2-bbf6786649e3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:24.662270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 39/71] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:24.663312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c8a54-6947-4872-a3a2-bbf6786649e3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:24.707055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:24.729514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a59d2-6946-419a-9a5d-a53e5fea24fd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:24.746825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:24.764653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1edba-136a-4c3c-a6d6-d906e19e38de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:24.765477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1edba-136a-4c3c-a6d6-d906e19e38de None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.765771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1edba-136a-4c3c-a6d6-d906e19e38de None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.768910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1edba-136a-4c3c-a6d6-d906e19e38de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.774389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c8a54-6947-4872-a3a2-bbf6786649e3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:24.777677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1edba-136a-4c3c-a6d6-d906e19e38de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-e3e1edba-136a-4c3c-a6d6-d906e19e38de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:24.786634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:24.787503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:24.788247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:24.789533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 33/72] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:01:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:24.789786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac202df-7b50-4a00-9fcb-05ef15c32b08 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.790358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac202df-7b50-4a00-9fcb-05ef15c32b08 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.792218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac202df-7b50-4a00-9fcb-05ef15c32b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.811809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8ac202df-7b50-4a00-9fcb-05ef15c32b08 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:24.813820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8ac202df-7b50-4a00-9fcb-05ef15c32b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:24.817341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0eb0a-2039-49c9-9ebe-eb59b2485a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:24.818024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0eb0a-2039-49c9-9ebe-eb59b2485a01 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.818316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0eb0a-2039-49c9-9ebe-eb59b2485a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.818632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0eb0a-2039-49c9-9ebe-eb59b2485a01 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.819391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0eb0a-2039-49c9-9ebe-eb59b2485a01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:24.819709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0eb0a-2039-49c9-9ebe-eb59b2485a01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:24.821128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0eb0a-2039-49c9-9ebe-eb59b2485a01 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:24.830237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 40/73] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:24.831201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0eb0a-2039-49c9-9ebe-eb59b2485a01 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:24.891801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 34/74] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:24.893545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42582a-6323-4681-858f-5f4d29debcda None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.893840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42582a-6323-4681-858f-5f4d29debcda None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.894123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42582a-6323-4681-858f-5f4d29debcda None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:24.897124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 35/75] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:24.897124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42582a-6323-4681-858f-5f4d29debcda None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:24.945158 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:24.951061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0eb0a-2039-49c9-9ebe-eb59b2485a01 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:24.990852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff02437-b5a4-47d8-b5b4-f9368c820ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:24.991656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff02437-b5a4-47d8-b5b4-f9368c820ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:24.991948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff02437-b5a4-47d8-b5b4-f9368c820ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:24.992232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff02437-b5a4-47d8-b5b4-f9368c820ea6 None admin] PATH_INFO: `/v3/users/d695ee7d689a4ca4ae7d161f5773ca60` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.006324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff02437-b5a4-47d8-b5b4-f9368c820ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=d695ee7d689a4ca4ae7d161f5773ca60)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:25.007837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff02437-b5a4-47d8-b5b4-f9368c820ea6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:25.009562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 41/76] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:24 2022] GET /identity/v3/users/d695ee7d689a4ca4ae7d161f5773ca60 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:25.010794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff02437-b5a4-47d8-b5b4-f9368c820ea6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:25.027103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42582a-6323-4681-858f-5f4d29debcda None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:25.035721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:25.064375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea9ac0-8182-4661-9241-4cd15c3219ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:25.065051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea9ac0-8182-4661-9241-4cd15c3219ca None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.065340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea9ac0-8182-4661-9241-4cd15c3219ca None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.065622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea9ac0-8182-4661-9241-4cd15c3219ca None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.066381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea9ac0-8182-4661-9241-4cd15c3219ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:25.066722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea9ac0-8182-4661-9241-4cd15c3219ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:25.068113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea9ac0-8182-4661-9241-4cd15c3219ca None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:25.077131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 36/77] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:01:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:25.079300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6093201-ed5c-4159-b541-2d4a66d5af8d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.079633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6093201-ed5c-4159-b541-2d4a66d5af8d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.079918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6093201-ed5c-4159-b541-2d4a66d5af8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.100441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b6093201-ed5c-4159-b541-2d4a66d5af8d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:25.101664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b6093201-ed5c-4159-b541-2d4a66d5af8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:25.118896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff02437-b5a4-47d8-b5b4-f9368c820ea6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:25.154528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16452450-5765-4d88-b7b5-3727d118ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:25.155472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16452450-5765-4d88-b7b5-3727d118ce8b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.155766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16452450-5765-4d88-b7b5-3727d118ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.156047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16452450-5765-4d88-b7b5-3727d118ce8b None admin] PATH_INFO: `/v3/users/87e06b5266a1425fb48c02407f951f73` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.157470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16452450-5765-4d88-b7b5-3727d118ce8b None admin] RBAC: Authorizing `identity:get_user(user_id=87e06b5266a1425fb48c02407f951f73)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:25.158788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16452450-5765-4d88-b7b5-3727d118ce8b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:25.160444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 42/78] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:25 2022] GET /identity/v3/users/87e06b5266a1425fb48c02407f951f73 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:25.161225 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16452450-5765-4d88-b7b5-3727d118ce8b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:25.179065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 37/79] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:25.179938 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6093201-ed5c-4159-b541-2d4a66d5af8d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:25.194243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:25.276950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16452450-5765-4d88-b7b5-3727d118ce8b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:25.293529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:25.298107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6093201-ed5c-4159-b541-2d4a66d5af8d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:25.315602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5f876-33cd-4528-b257-0a08faf90401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:25.316428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5f876-33cd-4528-b257-0a08faf90401 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.316716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5f876-33cd-4528-b257-0a08faf90401 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.317001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5f876-33cd-4528-b257-0a08faf90401 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.318393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5f876-33cd-4528-b257-0a08faf90401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:25.319900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5f876-33cd-4528-b257-0a08faf90401 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:25.321651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 43/80] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:25.322520 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5f876-33cd-4528-b257-0a08faf90401 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:25.341361 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d36e92-69ba-4156-8749-40cb640041f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:25.342048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d36e92-69ba-4156-8749-40cb640041f9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.342336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d36e92-69ba-4156-8749-40cb640041f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.342639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d36e92-69ba-4156-8749-40cb640041f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.419642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d36e92-69ba-4156-8749-40cb640041f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:25.421134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d36e92-69ba-4156-8749-40cb640041f9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:25.434595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:25.445869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5f876-33cd-4528-b257-0a08faf90401 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:25.482346 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099336c-b77e-438f-b220-24c19848b210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:25.483252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099336c-b77e-438f-b220-24c19848b210 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.483545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099336c-b77e-438f-b220-24c19848b210 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.483829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099336c-b77e-438f-b220-24c19848b210 None admin] PATH_INFO: `/v3/users/d695ee7d689a4ca4ae7d161f5773ca60` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.485147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099336c-b77e-438f-b220-24c19848b210 None admin] RBAC: Authorizing `identity:get_user(user_id=d695ee7d689a4ca4ae7d161f5773ca60)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:25.486638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099336c-b77e-438f-b220-24c19848b210 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:25.488453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 44/81] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:25 2022] GET /identity/v3/users/d695ee7d689a4ca4ae7d161f5773ca60 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:25.489306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099336c-b77e-438f-b220-24c19848b210 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:25.523136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:25.527382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 38/82] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:25.528134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d36e92-69ba-4156-8749-40cb640041f9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:25.596451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099336c-b77e-438f-b220-24c19848b210 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:25.630988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d36e92-69ba-4156-8749-40cb640041f9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:25.632169 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1237c25-335a-4230-bdfb-5211b56d09cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:25.632841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1237c25-335a-4230-bdfb-5211b56d09cb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.633825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1237c25-335a-4230-bdfb-5211b56d09cb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.633825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1237c25-335a-4230-bdfb-5211b56d09cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.659703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:25.664083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11718365-6bb3-4e4b-bb09-1a42e38fa5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:25.664890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11718365-6bb3-4e4b-bb09-1a42e38fa5ae None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.665183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11718365-6bb3-4e4b-bb09-1a42e38fa5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.665467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11718365-6bb3-4e4b-bb09-1a42e38fa5ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.676860 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11718365-6bb3-4e4b-bb09-1a42e38fa5ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-11718365-6bb3-4e4b-bb09-1a42e38fa5ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:25.684714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:25.685560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:25.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 39/83] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:25.686987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11718365-6bb3-4e4b-bb09-1a42e38fa5ae None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:25.706444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1237c25-335a-4230-bdfb-5211b56d09cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:25.707972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1237c25-335a-4230-bdfb-5211b56d09cb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:25.751494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:25.792063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11718365-6bb3-4e4b-bb09-1a42e38fa5ae None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:25.807959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 45/84] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:25.808829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1237c25-335a-4230-bdfb-5211b56d09cb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:25.833250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f667316-b315-4c6d-bc62-f79924cc621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:25.833933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f667316-b315-4c6d-bc62-f79924cc621a None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.834258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f667316-b315-4c6d-bc62-f79924cc621a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.834588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f667316-b315-4c6d-bc62-f79924cc621a None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.835358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f667316-b315-4c6d-bc62-f79924cc621a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:25.836869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f667316-b315-4c6d-bc62-f79924cc621a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:25.873463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:25.914227 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1237c25-335a-4230-bdfb-5211b56d09cb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:25.947314 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358320c2-8b73-411b-9e6a-2cbf4ed13061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:25.947965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358320c2-8b73-411b-9e6a-2cbf4ed13061 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:25.948255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358320c2-8b73-411b-9e6a-2cbf4ed13061 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:25.948536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358320c2-8b73-411b-9e6a-2cbf4ed13061 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:25.988739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 40/85] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:01:25 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:25.989577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f667316-b315-4c6d-bc62-f79924cc621a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.019983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358320c2-8b73-411b-9e6a-2cbf4ed13061 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.021510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358320c2-8b73-411b-9e6a-2cbf4ed13061 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.092075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:26.122282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f667316-b315-4c6d-bc62-f79924cc621a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.138446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:26.146257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 46/86] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:26.147182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358320c2-8b73-411b-9e6a-2cbf4ed13061 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.164891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe417-b339-4a51-95f3-bc61af4c849b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.165805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe417-b339-4a51-95f3-bc61af4c849b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.166127 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe417-b339-4a51-95f3-bc61af4c849b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.166467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32afe417-b339-4a51-95f3-bc61af4c849b None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.167998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afe417-b339-4a51-95f3-bc61af4c849b None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.169489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32afe417-b339-4a51-95f3-bc61af4c849b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.171376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 41/87] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:25 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:26.178217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe417-b339-4a51-95f3-bc61af4c849b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.275903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358320c2-8b73-411b-9e6a-2cbf4ed13061 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.303225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32afe417-b339-4a51-95f3-bc61af4c849b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.313889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e13efe-4d11-43d6-823f-ae70904d54e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.314602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e13efe-4d11-43d6-823f-ae70904d54e8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.314891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e13efe-4d11-43d6-823f-ae70904d54e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.315173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e13efe-4d11-43d6-823f-ae70904d54e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.315917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e13efe-4d11-43d6-823f-ae70904d54e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:26.316229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e13efe-4d11-43d6-823f-ae70904d54e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.317620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e13efe-4d11-43d6-823f-ae70904d54e8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.326697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 47/88] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:26.327298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e13efe-4d11-43d6-823f-ae70904d54e8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.353350 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f48ed-bb56-48c0-83f3-6319af67282c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.354141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f48ed-bb56-48c0-83f3-6319af67282c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.354448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f48ed-bb56-48c0-83f3-6319af67282c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.354737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f48ed-bb56-48c0-83f3-6319af67282c None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.355640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:26.355965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f48ed-bb56-48c0-83f3-6319af67282c None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.357409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f48ed-bb56-48c0-83f3-6319af67282c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.359310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 42/89] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:26.360049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f48ed-bb56-48c0-83f3-6319af67282c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.427137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:26.450303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e13efe-4d11-43d6-823f-ae70904d54e8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.487038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f48ed-bb56-48c0-83f3-6319af67282c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.488053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e225263-3c01-41eb-a159-b668ca66a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.489042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e225263-3c01-41eb-a159-b668ca66a602 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.489437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e225263-3c01-41eb-a159-b668ca66a602 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.489725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e225263-3c01-41eb-a159-b668ca66a602 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.490997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e225263-3c01-41eb-a159-b668ca66a602 None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.492457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e225263-3c01-41eb-a159-b668ca66a602 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.494167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 48/90] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:26.495175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e225263-3c01-41eb-a159-b668ca66a602 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.523479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d10e2-e0ea-47c6-b382-4bf68345ad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.524140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d10e2-e0ea-47c6-b382-4bf68345ad8e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.524429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d10e2-e0ea-47c6-b382-4bf68345ad8e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.524716 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d10e2-e0ea-47c6-b382-4bf68345ad8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.525559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d10e2-e0ea-47c6-b382-4bf68345ad8e None admin] RBAC: Adding query filter params (role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=adcc2a3589e7498db55bb58c9719de51, user.id=d695ee7d689a4ca4ae7d161f5773ca60) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:26.525884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d10e2-e0ea-47c6-b382-4bf68345ad8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.527370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d10e2-e0ea-47c6-b382-4bf68345ad8e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.533385 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 43/91] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/role_assignments?user.id=d695ee7d689a4ca4ae7d161f5773ca60&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:26.534245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d10e2-e0ea-47c6-b382-4bf68345ad8e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.658882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:26.677412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e225263-3c01-41eb-a159-b668ca66a602 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.730630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c30653-3330-4b26-a12a-de99d86275c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.731535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c30653-3330-4b26-a12a-de99d86275c7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.731820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c30653-3330-4b26-a12a-de99d86275c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.732097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c30653-3330-4b26-a12a-de99d86275c7 None admin] PATH_INFO: `/v3/users/d695ee7d689a4ca4ae7d161f5773ca60` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.733420 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c30653-3330-4b26-a12a-de99d86275c7 None admin] RBAC: Authorizing `identity:get_user(user_id=d695ee7d689a4ca4ae7d161f5773ca60)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.735864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c30653-3330-4b26-a12a-de99d86275c7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.744142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 49/92] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/users/d695ee7d689a4ca4ae7d161f5773ca60 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:26.745120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c30653-3330-4b26-a12a-de99d86275c7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.751822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d10e2-e0ea-47c6-b382-4bf68345ad8e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.767504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:26.791122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3884ce-5cea-4f1f-9bad-8fb845e3c73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.791373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3884ce-5cea-4f1f-9bad-8fb845e3c73c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.791668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3884ce-5cea-4f1f-9bad-8fb845e3c73c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.791948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3884ce-5cea-4f1f-9bad-8fb845e3c73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:26.793065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3884ce-5cea-4f1f-9bad-8fb845e3c73c None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=adcc2a3589e7498db55bb58c9719de51, user.id=87e06b5266a1425fb48c02407f951f73) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:26.793382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3884ce-5cea-4f1f-9bad-8fb845e3c73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:26.794916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3884ce-5cea-4f1f-9bad-8fb845e3c73c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:26.806442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 44/93] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/role_assignments?user.id=87e06b5266a1425fb48c02407f951f73&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=bd295cb5d1194b469beff0381c6b23af => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:26.806442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3884ce-5cea-4f1f-9bad-8fb845e3c73c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:26.946554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c30653-3330-4b26-a12a-de99d86275c7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.950283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3884ce-5cea-4f1f-9bad-8fb845e3c73c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:26.988158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf0be0-b9a2-49f7-8b82-085a8a7caf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:26.988822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf0be0-b9a2-49f7-8b82-085a8a7caf29 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:26.989105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf0be0-b9a2-49f7-8b82-085a8a7caf29 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:26.989389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf0be0-b9a2-49f7-8b82-085a8a7caf29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:27.028051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f4f1c-fb47-4a35-8a2e-e2b18ce034e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:27.029183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f4f1c-fb47-4a35-8a2e-e2b18ce034e5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:27.031132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f4f1c-fb47-4a35-8a2e-e2b18ce034e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:27.032185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f4f1c-fb47-4a35-8a2e-e2b18ce034e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:27.033841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f4f1c-fb47-4a35-8a2e-e2b18ce034e5 None admin] RBAC: Adding query filter params (role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=adcc2a3589e7498db55bb58c9719de51, user.id=d695ee7d689a4ca4ae7d161f5773ca60) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:27.036022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f4f1c-fb47-4a35-8a2e-e2b18ce034e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:27.042691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f4f1c-fb47-4a35-8a2e-e2b18ce034e5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:27.053689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:27.055113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 50/94] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/role_assignments?user.id=d695ee7d689a4ca4ae7d161f5773ca60&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:27.081989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf0be0-b9a2-49f7-8b82-085a8a7caf29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:27.083826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf0be0-b9a2-49f7-8b82-085a8a7caf29 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:27.203242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 45/95] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:27.209426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f4f1c-fb47-4a35-8a2e-e2b18ce034e5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:27.242027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:27.346610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f4f1c-fb47-4a35-8a2e-e2b18ce034e5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:27.390096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948302e-a96e-4d6e-b529-139cfd13040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:27.391919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948302e-a96e-4d6e-b529-139cfd13040d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:27.392389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948302e-a96e-4d6e-b529-139cfd13040d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:27.392832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948302e-a96e-4d6e-b529-139cfd13040d None admin] PATH_INFO: `/v3/projects/e0f2ea73517b4bb985689b50342c2cbe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:27.399348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948302e-a96e-4d6e-b529-139cfd13040d None admin] RBAC: Authorizing `identity:get_project(project_id=e0f2ea73517b4bb985689b50342c2cbe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:27.401058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5948302e-a96e-4d6e-b529-139cfd13040d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:27.403398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 51/96] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:27 2022] GET /identity/v3/projects/e0f2ea73517b4bb985689b50342c2cbe => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:27.409821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf0be0-b9a2-49f7-8b82-085a8a7caf29 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:27.511883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:27.535502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf0be0-b9a2-49f7-8b82-085a8a7caf29 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:27.577808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2fa2f-6047-4090-bd8b-14a1f936ca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:27.578787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2fa2f-6047-4090-bd8b-14a1f936ca96 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:27.579319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2fa2f-6047-4090-bd8b-14a1f936ca96 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:27.579428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2fa2f-6047-4090-bd8b-14a1f936ca96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:27.580273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2fa2f-6047-4090-bd8b-14a1f936ca96 None admin] RBAC: Adding query filter params (role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=e0f2ea73517b4bb985689b50342c2cbe, user.id=d695ee7d689a4ca4ae7d161f5773ca60) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:27.580597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2fa2f-6047-4090-bd8b-14a1f936ca96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:27.582062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2fa2f-6047-4090-bd8b-14a1f936ca96 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:27.588973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 46/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:27 2022] GET /identity/v3/role_assignments?user.id=d695ee7d689a4ca4ae7d161f5773ca60&scope.project.id=e0f2ea73517b4bb985689b50342c2cbe&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:29.263388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66660a57-0755-433a-b3e5-ac9a160c56da None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.264406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66660a57-0755-433a-b3e5-ac9a160c56da None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.264902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66660a57-0755-433a-b3e5-ac9a160c56da None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.267150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 52/98] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:29.273375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dc435-d524-4cef-acf9-7f4a1b72fcbe None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.273868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dc435-d524-4cef-acf9-7f4a1b72fcbe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.273961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dc435-d524-4cef-acf9-7f4a1b72fcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.297511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ea2dc435-d524-4cef-acf9-7f4a1b72fcbe None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:29.298658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ea2dc435-d524-4cef-acf9-7f4a1b72fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:29.385234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 47/99] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:29.421321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66660a57-0755-433a-b3e5-ac9a160c56da None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:29.543917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:29.565370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66660a57-0755-433a-b3e5-ac9a160c56da None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:29.606532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b65ad-ebe0-4024-8f65-6d6250d64ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:29.607347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b65ad-ebe0-4024-8f65-6d6250d64ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.607694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b65ad-ebe0-4024-8f65-6d6250d64ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.607976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b65ad-ebe0-4024-8f65-6d6250d64ac7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.609196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b65ad-ebe0-4024-8f65-6d6250d64ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:29.611953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b65ad-ebe0-4024-8f65-6d6250d64ac7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:29.614077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 53/100] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:29.619261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dc435-d524-4cef-acf9-7f4a1b72fcbe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:29.682315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ac9a3-242e-4810-8001-3c2d18765398 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.682999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ac9a3-242e-4810-8001-3c2d18765398 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.683518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ac9a3-242e-4810-8001-3c2d18765398 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.685074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 54/101] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:29.691457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23253768-ff0d-46c5-91ab-2b5a1a8cc336 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.692019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23253768-ff0d-46c5-91ab-2b5a1a8cc336 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.692504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23253768-ff0d-46c5-91ab-2b5a1a8cc336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.716060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:29.736428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-23253768-ff0d-46c5-91ab-2b5a1a8cc336 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:29.737997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-23253768-ff0d-46c5-91ab-2b5a1a8cc336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:29.749325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dc435-d524-4cef-acf9-7f4a1b72fcbe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:29.802688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87354ad8-8c9f-4ac7-aa62-11f4f0dc877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:29.804930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87354ad8-8c9f-4ac7-aa62-11f4f0dc877c None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:29.805555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87354ad8-8c9f-4ac7-aa62-11f4f0dc877c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:29.806740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87354ad8-8c9f-4ac7-aa62-11f4f0dc877c None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:29.808150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87354ad8-8c9f-4ac7-aa62-11f4f0dc877c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:29.810001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87354ad8-8c9f-4ac7-aa62-11f4f0dc877c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:29.878819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 55/102] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 12:01:29.913702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23253768-ff0d-46c5-91ab-2b5a1a8cc336 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:29.931627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 48/103] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:01:29 2022] POST /identity/v3/projects => generated 302 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:30.070925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:30.088250 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23253768-ff0d-46c5-91ab-2b5a1a8cc336 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.090208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff64a9-4180-40fc-932d-a9ad57371841 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.090525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff64a9-4180-40fc-932d-a9ad57371841 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.091028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff64a9-4180-40fc-932d-a9ad57371841 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.092655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 49/104] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:30.119989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52846ca-694e-4ae9-8c80-613223de7e0c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.120617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52846ca-694e-4ae9-8c80-613223de7e0c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.121129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52846ca-694e-4ae9-8c80-613223de7e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.135427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb2da2-42bf-4731-9e3f-f5723cf04ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.136187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb2da2-42bf-4731-9e3f-f5723cf04ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.136478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb2da2-42bf-4731-9e3f-f5723cf04ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.136759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb2da2-42bf-4731-9e3f-f5723cf04ca2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.161849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b52846ca-694e-4ae9-8c80-613223de7e0c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:30.163091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b52846ca-694e-4ae9-8c80-613223de7e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:30.167066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb2da2-42bf-4731-9e3f-f5723cf04ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-91bb2da2-42bf-4731-9e3f-f5723cf04ca2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:30.183804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:30.184802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:30.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 56/105] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:30.191994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb2da2-42bf-4731-9e3f-f5723cf04ca2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.308445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 50/106] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:30.310356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b049656-60dd-4a0f-b48a-6138365a94da None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.310820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b049656-60dd-4a0f-b48a-6138365a94da None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.311275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b049656-60dd-4a0f-b48a-6138365a94da None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.313671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 51/107] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:30.323944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3d7bd-c483-4797-96a8-9ad0e2942baa None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.324462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3d7bd-c483-4797-96a8-9ad0e2942baa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.324896 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3d7bd-c483-4797-96a8-9ad0e2942baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.350813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb2da2-42bf-4731-9e3f-f5723cf04ca2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.353540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-87e3d7bd-c483-4797-96a8-9ad0e2942baa None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:30.354905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-87e3d7bd-c483-4797-96a8-9ad0e2942baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:30.397907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc42774-37b5-4f00-a28d-d17f0d954663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.397907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc42774-37b5-4f00-a28d-d17f0d954663 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.397907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc42774-37b5-4f00-a28d-d17f0d954663 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.397907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc42774-37b5-4f00-a28d-d17f0d954663 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.398321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc42774-37b5-4f00-a28d-d17f0d954663 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:30.398321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc42774-37b5-4f00-a28d-d17f0d954663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:30.398321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc42774-37b5-4f00-a28d-d17f0d954663 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:30.408392 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 57/108] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:30.409968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc42774-37b5-4f00-a28d-d17f0d954663 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.420643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:30.426051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:30.442344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 52/109] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:30.443295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3d7bd-c483-4797-96a8-9ad0e2942baa None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.530560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc42774-37b5-4f00-a28d-d17f0d954663 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.565279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3d7bd-c483-4797-96a8-9ad0e2942baa None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.567067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d1e85-2997-4e76-89df-5764190b4b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.568057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d1e85-2997-4e76-89df-5764190b4b7d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.568483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d1e85-2997-4e76-89df-5764190b4b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.568853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d1e85-2997-4e76-89df-5764190b4b7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.582058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d1e85-2997-4e76-89df-5764190b4b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-687d1e85-2997-4e76-89df-5764190b4b7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:30.592764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:30.593660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 58/110] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:30.595627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d924-c329-4f4d-9205-ca296b098be7 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.596296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d924-c329-4f4d-9205-ca296b098be7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.596296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02d924-c329-4f4d-9205-ca296b098be7 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.596296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 59/111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:30.601799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d924-c329-4f4d-9205-ca296b098be7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.619858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153d3c3-bff5-401a-8b52-c2a0198ffa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.620879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153d3c3-bff5-401a-8b52-c2a0198ffa4c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.621274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153d3c3-bff5-401a-8b52-c2a0198ffa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.621582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153d3c3-bff5-401a-8b52-c2a0198ffa4c None admin] PATH_INFO: `/v3/users/a32fb42b027d40c08da7b59f1d60dc77` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.648959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153d3c3-bff5-401a-8b52-c2a0198ffa4c None admin] RBAC: Authorizing `identity:get_user(user_id=a32fb42b027d40c08da7b59f1d60dc77)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:30.650944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153d3c3-bff5-401a-8b52-c2a0198ffa4c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:30.653625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 53/112] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/users/a32fb42b027d40c08da7b59f1d60dc77 => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:30.655096 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1f527-8a77-4120-a4eb-ca6b7c55dcc2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.655473 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1f527-8a77-4120-a4eb-ca6b7c55dcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.655813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1f527-8a77-4120-a4eb-ca6b7c55dcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.682459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-88e1f527-8a77-4120-a4eb-ca6b7c55dcc2 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:30.683834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-88e1f527-8a77-4120-a4eb-ca6b7c55dcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:30.745889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:30.749603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:30.797034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 54/113] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:30.798455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1f527-8a77-4120-a4eb-ca6b7c55dcc2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.807859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02d924-c329-4f4d-9205-ca296b098be7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.864890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22043806-96a2-4dc7-879f-b6e559035816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.865714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22043806-96a2-4dc7-879f-b6e559035816 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.866003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22043806-96a2-4dc7-879f-b6e559035816 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.866315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22043806-96a2-4dc7-879f-b6e559035816 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.880334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22043806-96a2-4dc7-879f-b6e559035816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-22043806-96a2-4dc7-879f-b6e559035816 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:30.888374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:30.889309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 60/114] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:30.890558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22043806-96a2-4dc7-879f-b6e559035816 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:30.942792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1f527-8a77-4120-a4eb-ca6b7c55dcc2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:30.988519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502730e8-90b9-4f0e-b892-2a5a73e81883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:30.989203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502730e8-90b9-4f0e-b892-2a5a73e81883 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:30.989490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502730e8-90b9-4f0e-b892-2a5a73e81883 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:30.989772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502730e8-90b9-4f0e-b892-2a5a73e81883 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:30.990683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502730e8-90b9-4f0e-b892-2a5a73e81883 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:30.991100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502730e8-90b9-4f0e-b892-2a5a73e81883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:30.992514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502730e8-90b9-4f0e-b892-2a5a73e81883 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.002482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 55/115] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:31.003324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502730e8-90b9-4f0e-b892-2a5a73e81883 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.031137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22043806-96a2-4dc7-879f-b6e559035816 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.040618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.068884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.080976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cee7a4-2de9-4996-93bd-e530478a3814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.081644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cee7a4-2de9-4996-93bd-e530478a3814 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.081933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cee7a4-2de9-4996-93bd-e530478a3814 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.082211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cee7a4-2de9-4996-93bd-e530478a3814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.129776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502730e8-90b9-4f0e-b892-2a5a73e81883 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.161816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cee7a4-2de9-4996-93bd-e530478a3814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.163498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cee7a4-2de9-4996-93bd-e530478a3814 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.170111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c5065-588a-4b53-ae42-36475df4d0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.170942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5065-588a-4b53-ae42-36475df4d0ae None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.171243 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5065-588a-4b53-ae42-36475df4d0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.171515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5065-588a-4b53-ae42-36475df4d0ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.186806 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c5065-588a-4b53-ae42-36475df4d0ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b47c5065-588a-4b53-ae42-36475df4d0ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:31.198980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:31.199835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:31.200663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 56/116] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:31.201277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c5065-588a-4b53-ae42-36475df4d0ae None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.272960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 61/117] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:31.273683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cee7a4-2de9-4996-93bd-e530478a3814 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.306490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.307261 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.313534 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c5065-588a-4b53-ae42-36475df4d0ae None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.354327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf72ce-ee53-458d-a0dd-7345feccfad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.355028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf72ce-ee53-458d-a0dd-7345feccfad9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.355327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf72ce-ee53-458d-a0dd-7345feccfad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.355607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf72ce-ee53-458d-a0dd-7345feccfad9 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.356365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf72ce-ee53-458d-a0dd-7345feccfad9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:31.356705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf72ce-ee53-458d-a0dd-7345feccfad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.358154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf72ce-ee53-458d-a0dd-7345feccfad9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.367526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 57/118] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:31.373337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2f94-d9c9-488b-bdeb-82b229437ce3 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.373693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2f94-d9c9-488b-bdeb-82b229437ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.373985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2f94-d9c9-488b-bdeb-82b229437ce3 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.378450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 58/119] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:31.378450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b2f94-d9c9-488b-bdeb-82b229437ce3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.384897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cee7a4-2de9-4996-93bd-e530478a3814 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.421130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c634a-ce85-41a7-83bd-3ef35b033fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.422706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c634a-ce85-41a7-83bd-3ef35b033fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.423260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c634a-ce85-41a7-83bd-3ef35b033fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.423546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c634a-ce85-41a7-83bd-3ef35b033fc0 None admin] PATH_INFO: `/v3/users/d695ee7d689a4ca4ae7d161f5773ca60` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.424770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c634a-ce85-41a7-83bd-3ef35b033fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=d695ee7d689a4ca4ae7d161f5773ca60)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.426224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c634a-ce85-41a7-83bd-3ef35b033fc0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.427888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 62/120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/users/d695ee7d689a4ca4ae7d161f5773ca60 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:31.432514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c634a-ce85-41a7-83bd-3ef35b033fc0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.494853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b2f94-d9c9-488b-bdeb-82b229437ce3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.536175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e1eb1-dbda-4105-9020-df48ae93ee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.536843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e1eb1-dbda-4105-9020-df48ae93ee98 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.537137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e1eb1-dbda-4105-9020-df48ae93ee98 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.537463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e1eb1-dbda-4105-9020-df48ae93ee98 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.538272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e1eb1-dbda-4105-9020-df48ae93ee98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:31.538660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e1eb1-dbda-4105-9020-df48ae93ee98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.540007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e1eb1-dbda-4105-9020-df48ae93ee98 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.548402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.556057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 59/121] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:31.556057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e1eb1-dbda-4105-9020-df48ae93ee98 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.560753 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c634a-ce85-41a7-83bd-3ef35b033fc0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.576138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.598593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938d362-2dad-44ae-881c-2978d354fee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.599453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938d362-2dad-44ae-881c-2978d354fee6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.599743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938d362-2dad-44ae-881c-2978d354fee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.600039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938d362-2dad-44ae-881c-2978d354fee6 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.601262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938d362-2dad-44ae-881c-2978d354fee6 None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.602775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938d362-2dad-44ae-881c-2978d354fee6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.604550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 63/122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:31.605724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89e711-3081-455c-9f02-d3a38b925a5b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.606015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89e711-3081-455c-9f02-d3a38b925a5b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.606301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89e711-3081-455c-9f02-d3a38b925a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.624596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9e89e711-3081-455c-9f02-d3a38b925a5b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:31.625846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9e89e711-3081-455c-9f02-d3a38b925a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:31.674037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e1eb1-dbda-4105-9020-df48ae93ee98 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.697347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 64/123] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:31.706565 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89e711-3081-455c-9f02-d3a38b925a5b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.712227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273e5d-96e8-4a98-a553-2c1e3eb06ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.713208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273e5d-96e8-4a98-a553-2c1e3eb06ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.713500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273e5d-96e8-4a98-a553-2c1e3eb06ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.713784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e273e5d-96e8-4a98-a553-2c1e3eb06ae5 None admin] PATH_INFO: `/v3/users/87e06b5266a1425fb48c02407f951f73` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.715080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e273e5d-96e8-4a98-a553-2c1e3eb06ae5 None admin] RBAC: Authorizing `identity:get_user(user_id=87e06b5266a1425fb48c02407f951f73)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.716767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e273e5d-96e8-4a98-a553-2c1e3eb06ae5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.718497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 60/124] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/users/87e06b5266a1425fb48c02407f951f73 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:31.724041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273e5d-96e8-4a98-a553-2c1e3eb06ae5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.820327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.824354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:31.831803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89e711-3081-455c-9f02-d3a38b925a5b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.835369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e273e5d-96e8-4a98-a553-2c1e3eb06ae5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.867179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf5854-79b4-4a6b-9ce7-65fed9d86ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.868057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf5854-79b4-4a6b-9ce7-65fed9d86ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.868351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf5854-79b4-4a6b-9ce7-65fed9d86ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.868636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf5854-79b4-4a6b-9ce7-65fed9d86ea0 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.869834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf5854-79b4-4a6b-9ce7-65fed9d86ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.871545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf5854-79b4-4a6b-9ce7-65fed9d86ea0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.873399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 65/125] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:31.875622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49076bf-9a0e-4a1a-b046-849d988e1a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:31.876968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49076bf-9a0e-4a1a-b046-849d988e1a33 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:31.877480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49076bf-9a0e-4a1a-b046-849d988e1a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:31.877934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49076bf-9a0e-4a1a-b046-849d988e1a33 None admin] PATH_INFO: `/v3/users/d695ee7d689a4ca4ae7d161f5773ca60` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:31.882008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf5854-79b4-4a6b-9ce7-65fed9d86ea0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.882115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49076bf-9a0e-4a1a-b046-849d988e1a33 None admin] RBAC: Authorizing `identity:get_user(user_id=d695ee7d689a4ca4ae7d161f5773ca60)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:31.882115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49076bf-9a0e-4a1a-b046-849d988e1a33 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:31.882977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 61/126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/users/d695ee7d689a4ca4ae7d161f5773ca60 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:31.883910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49076bf-9a0e-4a1a-b046-849d988e1a33 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:31.987906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf5854-79b4-4a6b-9ce7-65fed9d86ea0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:31.994902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49076bf-9a0e-4a1a-b046-849d988e1a33 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:32.035117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705937f7-fb49-46e2-ae1c-0a2eb973590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:32.035775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705937f7-fb49-46e2-ae1c-0a2eb973590a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.036061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705937f7-fb49-46e2-ae1c-0a2eb973590a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.036159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80005a-58e6-4754-84a6-1819db7b8138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:32.036331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705937f7-fb49-46e2-ae1c-0a2eb973590a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.036936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80005a-58e6-4754-84a6-1819db7b8138 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.037165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705937f7-fb49-46e2-ae1c-0a2eb973590a None admin] RBAC: Adding query filter params (role.id=6014fae253be445d9675cf76770c7b4c, scope.project.id=adcc2a3589e7498db55bb58c9719de51, user.id=a32fb42b027d40c08da7b59f1d60dc77) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:32.037265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80005a-58e6-4754-84a6-1819db7b8138 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.037479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705937f7-fb49-46e2-ae1c-0a2eb973590a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:32.037567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80005a-58e6-4754-84a6-1819db7b8138 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.038976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705937f7-fb49-46e2-ae1c-0a2eb973590a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:32.054720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 66/127] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/role_assignments?user.id=a32fb42b027d40c08da7b59f1d60dc77&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=6014fae253be445d9675cf76770c7b4c => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:32.054720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705937f7-fb49-46e2-ae1c-0a2eb973590a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:32.055000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80005a-58e6-4754-84a6-1819db7b8138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1f80005a-58e6-4754-84a6-1819db7b8138 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:32.056905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:32.057720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:32.058498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:32.063482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 62/128] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:32.064616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80005a-58e6-4754-84a6-1819db7b8138 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:32.084755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:32.106839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:32.186267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80005a-58e6-4754-84a6-1819db7b8138 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:32.214299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705937f7-fb49-46e2-ae1c-0a2eb973590a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:32.230718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553599a7-5367-498d-9b65-32f49bf1d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:32.232148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553599a7-5367-498d-9b65-32f49bf1d1dc None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.232437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553599a7-5367-498d-9b65-32f49bf1d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.232723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553599a7-5367-498d-9b65-32f49bf1d1dc None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51/users/d695ee7d689a4ca4ae7d161f5773ca60/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.236798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553599a7-5367-498d-9b65-32f49bf1d1dc None admin] RBAC: Authorizing `identity:create_grant(project_id=adcc2a3589e7498db55bb58c9719de51, user_id=d695ee7d689a4ca4ae7d161f5773ca60, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:32.238295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553599a7-5367-498d-9b65-32f49bf1d1dc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:32.256704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8a539-449d-498a-b15f-0fcbf1f315ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:32.257967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8a539-449d-498a-b15f-0fcbf1f315ab None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.258259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8a539-449d-498a-b15f-0fcbf1f315ab None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.258596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8a539-449d-498a-b15f-0fcbf1f315ab None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.259918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8a539-449d-498a-b15f-0fcbf1f315ab None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:32.261449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8a539-449d-498a-b15f-0fcbf1f315ab None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:32.263217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 67/129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:32 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:32.263999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8a539-449d-498a-b15f-0fcbf1f315ab None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:32.370982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 63/130] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:01:32 2022] PUT /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51/users/d695ee7d689a4ca4ae7d161f5773ca60/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:32.376759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553599a7-5367-498d-9b65-32f49bf1d1dc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:32.378388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:32.391985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8a539-449d-498a-b15f-0fcbf1f315ab None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:32.444913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d4be8-074d-4f01-a755-748559fe47ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:32.446871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d4be8-074d-4f01-a755-748559fe47ef None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.447484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d4be8-074d-4f01-a755-748559fe47ef None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.447974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d4be8-074d-4f01-a755-748559fe47ef None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.450634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d4be8-074d-4f01-a755-748559fe47ef None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:32.453026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d4be8-074d-4f01-a755-748559fe47ef None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:32.457086 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 68/131] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:32 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:32.463230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d4be8-074d-4f01-a755-748559fe47ef None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:32.488364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:32.529893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553599a7-5367-498d-9b65-32f49bf1d1dc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:32.599459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce96370-6fcc-45de-bc7c-09a444c614e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:32.600780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce96370-6fcc-45de-bc7c-09a444c614e6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.601409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce96370-6fcc-45de-bc7c-09a444c614e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.602063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce96370-6fcc-45de-bc7c-09a444c614e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.603207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce96370-6fcc-45de-bc7c-09a444c614e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:32.603784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce96370-6fcc-45de-bc7c-09a444c614e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:32.606519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce96370-6fcc-45de-bc7c-09a444c614e6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:32.617673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 64/132] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:32.623842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce96370-6fcc-45de-bc7c-09a444c614e6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:32.705495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d4be8-074d-4f01-a755-748559fe47ef None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:32.765979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113f13a-59af-4846-9031-7e483d1ef766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:32.766976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce96370-6fcc-45de-bc7c-09a444c614e6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:32.767391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113f13a-59af-4846-9031-7e483d1ef766 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.767816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113f13a-59af-4846-9031-7e483d1ef766 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.768230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113f13a-59af-4846-9031-7e483d1ef766 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51/users/87e06b5266a1425fb48c02407f951f73/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.771759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113f13a-59af-4846-9031-7e483d1ef766 None admin] RBAC: Authorizing `identity:create_grant(project_id=adcc2a3589e7498db55bb58c9719de51, user_id=87e06b5266a1425fb48c02407f951f73, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:32.773383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113f13a-59af-4846-9031-7e483d1ef766 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:32.806022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aa09f-b6be-4341-858e-000669309e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:32.807191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aa09f-b6be-4341-858e-000669309e37 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.807600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aa09f-b6be-4341-858e-000669309e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.808032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aa09f-b6be-4341-858e-000669309e37 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51/users/d695ee7d689a4ca4ae7d161f5773ca60/roles/13607cfcced34c8badf88b8c244d32c5` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.810544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aa09f-b6be-4341-858e-000669309e37 None admin] RBAC: Authorizing `identity:create_grant(project_id=adcc2a3589e7498db55bb58c9719de51, user_id=d695ee7d689a4ca4ae7d161f5773ca60, role_id=13607cfcced34c8badf88b8c244d32c5)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:32.812120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aa09f-b6be-4341-858e-000669309e37 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:32.817003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:32.909669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 69/133] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:01:32 2022] PUT /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51/users/87e06b5266a1425fb48c02407f951f73/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:32.910878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113f13a-59af-4846-9031-7e483d1ef766 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:32.943556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 65/134] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:01:32 2022] PUT /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51/users/d695ee7d689a4ca4ae7d161f5773ca60/roles/13607cfcced34c8badf88b8c244d32c5 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:32.946668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73413249-b3c4-4592-92ab-5fbaf9368a13 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.947530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73413249-b3c4-4592-92ab-5fbaf9368a13 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.949590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73413249-b3c4-4592-92ab-5fbaf9368a13 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.954118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 66/135] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:32.962032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490742f4-92c3-4b3c-834c-86a102c124fc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:32.963778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490742f4-92c3-4b3c-834c-86a102c124fc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:32.964319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490742f4-92c3-4b3c-834c-86a102c124fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:32.974624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:33.000438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-490742f4-92c3-4b3c-834c-86a102c124fc None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:33.012443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-490742f4-92c3-4b3c-834c-86a102c124fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:33.209013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 67/136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:33.216584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113f13a-59af-4846-9031-7e483d1ef766 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:33.271262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f016a-1f15-413a-b7d5-c167618ed638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:33.272540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f016a-1f15-413a-b7d5-c167618ed638 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:33.273035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f016a-1f15-413a-b7d5-c167618ed638 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:33.273434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490742f4-92c3-4b3c-834c-86a102c124fc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:33.273735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f016a-1f15-413a-b7d5-c167618ed638 None admin] PATH_INFO: `/v3/users/d695ee7d689a4ca4ae7d161f5773ca60` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:33.275680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f016a-1f15-413a-b7d5-c167618ed638 None admin] RBAC: Authorizing `identity:get_user(user_id=d695ee7d689a4ca4ae7d161f5773ca60)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:33.277386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f016a-1f15-413a-b7d5-c167618ed638 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:33.279322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 70/137] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:32 2022] GET /identity/v3/users/d695ee7d689a4ca4ae7d161f5773ca60 => generated 275 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:33.286550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f016a-1f15-413a-b7d5-c167618ed638 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:33.397378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:33.410606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f016a-1f15-413a-b7d5-c167618ed638 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:33.469636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f8096-3f83-47df-8b37-9a962c4a1930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:33.471347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f8096-3f83-47df-8b37-9a962c4a1930 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:33.472188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f8096-3f83-47df-8b37-9a962c4a1930 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:33.472959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f8096-3f83-47df-8b37-9a962c4a1930 None admin] PATH_INFO: `/v3/projects/e0f2ea73517b4bb985689b50342c2cbe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:33.476084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f8096-3f83-47df-8b37-9a962c4a1930 None admin] RBAC: Authorizing `identity:get_project(project_id=e0f2ea73517b4bb985689b50342c2cbe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:33.478766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f8096-3f83-47df-8b37-9a962c4a1930 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:33.481948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 71/138] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:33 2022] GET /identity/v3/projects/e0f2ea73517b4bb985689b50342c2cbe => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:33.489708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f8096-3f83-47df-8b37-9a962c4a1930 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:33.561408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490742f4-92c3-4b3c-834c-86a102c124fc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:33.616222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f8096-3f83-47df-8b37-9a962c4a1930 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:33.624291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:33.655138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426b9da-6636-4700-953e-92cd12e4bc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:33.655955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426b9da-6636-4700-953e-92cd12e4bc62 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:33.656879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426b9da-6636-4700-953e-92cd12e4bc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:33.657158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426b9da-6636-4700-953e-92cd12e4bc62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:33.658401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426b9da-6636-4700-953e-92cd12e4bc62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:33.659847 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426b9da-6636-4700-953e-92cd12e4bc62 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:33.660799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa620f72-851d-48f1-85d0-90be447b78b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:33.661472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 68/139] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:33.662145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa620f72-851d-48f1-85d0-90be447b78b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:33.662711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa620f72-851d-48f1-85d0-90be447b78b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:33.663191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa620f72-851d-48f1-85d0-90be447b78b7 None admin] PATH_INFO: `/v3/projects/e0f2ea73517b4bb985689b50342c2cbe/users/d695ee7d689a4ca4ae7d161f5773ca60/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:33.665454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa620f72-851d-48f1-85d0-90be447b78b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0f2ea73517b4bb985689b50342c2cbe, user_id=d695ee7d689a4ca4ae7d161f5773ca60, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:33.666721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426b9da-6636-4700-953e-92cd12e4bc62 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:33.667882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa620f72-851d-48f1-85d0-90be447b78b7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:33.766622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 72/140] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:01:33 2022] PUT /identity/v3/projects/e0f2ea73517b4bb985689b50342c2cbe/users/d695ee7d689a4ca4ae7d161f5773ca60/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:01:33.842733 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426b9da-6636-4700-953e-92cd12e4bc62 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:33.899930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765d00e-0823-477e-a1ae-f0288256a4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:33.900633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765d00e-0823-477e-a1ae-f0288256a4d8 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:33.900924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765d00e-0823-477e-a1ae-f0288256a4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:33.901211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765d00e-0823-477e-a1ae-f0288256a4d8 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:33.901965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765d00e-0823-477e-a1ae-f0288256a4d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:33.903574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765d00e-0823-477e-a1ae-f0288256a4d8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:34.037587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 69/141] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:01:33 2022] POST /identity/v3/users => generated 283 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:35.110736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc86d5-b149-4d48-be1d-08ad7386e473 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.110736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc86d5-b149-4d48-be1d-08ad7386e473 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.110736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc86d5-b149-4d48-be1d-08ad7386e473 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.111713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 73/142] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:35.121454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2123bf3-18eb-46ee-83d3-b11d7e39b380 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.121926 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2123bf3-18eb-46ee-83d3-b11d7e39b380 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.122007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2123bf3-18eb-46ee-83d3-b11d7e39b380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.147525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b2123bf3-18eb-46ee-83d3-b11d7e39b380 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:35.148759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b2123bf3-18eb-46ee-83d3-b11d7e39b380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:35.206923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:35.230857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 70/143] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:35.269239 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc86d5-b149-4d48-be1d-08ad7386e473 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:35.299263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:35.411495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc86d5-b149-4d48-be1d-08ad7386e473 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:35.452617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17206e-340d-4743-a1f9-602ca309515e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:35.456698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17206e-340d-4743-a1f9-602ca309515e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.457289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17206e-340d-4743-a1f9-602ca309515e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.457766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17206e-340d-4743-a1f9-602ca309515e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.469845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17206e-340d-4743-a1f9-602ca309515e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-be17206e-340d-4743-a1f9-602ca309515e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:35.478222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:35.479135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:35.479864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:35.482194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 74/144] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:35.491623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2123bf3-18eb-46ee-83d3-b11d7e39b380 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:35.604998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9210bc-4f3d-4374-8469-dd9ba1862a93 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.605682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9210bc-4f3d-4374-8469-dd9ba1862a93 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.606234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9210bc-4f3d-4374-8469-dd9ba1862a93 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.607821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 75/145] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:35.613706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4e84a-3241-4116-9bb2-fcca0a8d9594 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.614302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4e84a-3241-4116-9bb2-fcca0a8d9594 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.614450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2123bf3-18eb-46ee-83d3-b11d7e39b380 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:35.614932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4e84a-3241-4116-9bb2-fcca0a8d9594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.642668 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f2b4e84a-3241-4116-9bb2-fcca0a8d9594 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:35.644210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f2b4e84a-3241-4116-9bb2-fcca0a8d9594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:35.664180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de193e9-a236-4495-b91d-d62191c2cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:35.664873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de193e9-a236-4495-b91d-d62191c2cf5f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.665166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de193e9-a236-4495-b91d-d62191c2cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.665451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de193e9-a236-4495-b91d-d62191c2cf5f None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.666265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de193e9-a236-4495-b91d-d62191c2cf5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:35.669571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de193e9-a236-4495-b91d-d62191c2cf5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:35.669571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de193e9-a236-4495-b91d-d62191c2cf5f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:35.686431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 71/146] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:35.691484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de193e9-a236-4495-b91d-d62191c2cf5f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:35.751744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:35.754479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 76/147] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:35.790689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4e84a-3241-4116-9bb2-fcca0a8d9594 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:35.807008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:35.827728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de193e9-a236-4495-b91d-d62191c2cf5f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:35.875698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0387075-9775-4b23-befd-3346d8ebafc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:35.876818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0387075-9775-4b23-befd-3346d8ebafc6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:35.877108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0387075-9775-4b23-befd-3346d8ebafc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:35.877389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0387075-9775-4b23-befd-3346d8ebafc6 None admin] PATH_INFO: `/v3/users/a32fb42b027d40c08da7b59f1d60dc77` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:35.880299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0387075-9775-4b23-befd-3346d8ebafc6 None admin] RBAC: Authorizing `identity:get_user(user_id=a32fb42b027d40c08da7b59f1d60dc77)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:35.881780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0387075-9775-4b23-befd-3346d8ebafc6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:35.883513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 72/148] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:35 2022] GET /identity/v3/users/a32fb42b027d40c08da7b59f1d60dc77 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:35.889266 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0387075-9775-4b23-befd-3346d8ebafc6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:36.044062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0387075-9775-4b23-befd-3346d8ebafc6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:36.045274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4e84a-3241-4116-9bb2-fcca0a8d9594 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:36.064711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:36.103154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a620e-568d-4af6-82f6-01c9e236ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:36.104044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a620e-568d-4af6-82f6-01c9e236ba09 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:36.104332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a620e-568d-4af6-82f6-01c9e236ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:36.104612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a620e-568d-4af6-82f6-01c9e236ba09 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:36.105857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a620e-568d-4af6-82f6-01c9e236ba09 None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:36.107384 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a620e-568d-4af6-82f6-01c9e236ba09 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:36.110175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 73/149] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:35 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:36.115993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a620e-568d-4af6-82f6-01c9e236ba09 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:36.124128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43935b4-5f9e-4add-8083-2d5b9acdb48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:36.126084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43935b4-5f9e-4add-8083-2d5b9acdb48e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:36.126339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43935b4-5f9e-4add-8083-2d5b9acdb48e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:36.126584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43935b4-5f9e-4add-8083-2d5b9acdb48e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:36.147574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43935b4-5f9e-4add-8083-2d5b9acdb48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e43935b4-5f9e-4add-8083-2d5b9acdb48e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:36.155368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:36.156598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:36.157615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 77/150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:36.162814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43935b4-5f9e-4add-8083-2d5b9acdb48e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:36.271400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:36.277029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a620e-568d-4af6-82f6-01c9e236ba09 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:36.308634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43935b4-5f9e-4add-8083-2d5b9acdb48e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:36.332513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07980e21-700a-4cfe-8024-573d7e52de23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:36.333449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980e21-700a-4cfe-8024-573d7e52de23 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:36.333739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980e21-700a-4cfe-8024-573d7e52de23 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:36.334022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980e21-700a-4cfe-8024-573d7e52de23 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51/users/a32fb42b027d40c08da7b59f1d60dc77/roles/6014fae253be445d9675cf76770c7b4c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:36.339811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07980e21-700a-4cfe-8024-573d7e52de23 None admin] RBAC: Authorizing `identity:create_grant(project_id=adcc2a3589e7498db55bb58c9719de51, user_id=a32fb42b027d40c08da7b59f1d60dc77, role_id=6014fae253be445d9675cf76770c7b4c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:36.341290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07980e21-700a-4cfe-8024-573d7e52de23 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:36.359857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177fee0-1008-4029-bb34-84cde7b9f404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:36.360610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177fee0-1008-4029-bb34-84cde7b9f404 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:36.361045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177fee0-1008-4029-bb34-84cde7b9f404 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:36.361327 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177fee0-1008-4029-bb34-84cde7b9f404 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:36.362249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177fee0-1008-4029-bb34-84cde7b9f404 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:36.362591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177fee0-1008-4029-bb34-84cde7b9f404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:36.364020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177fee0-1008-4029-bb34-84cde7b9f404 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:36.376980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 78/151] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:36.378304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b436-7185-4057-b777-a3d58c1f2122 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:36.378658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b436-7185-4057-b777-a3d58c1f2122 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:36.378949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b436-7185-4057-b777-a3d58c1f2122 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:36.380175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 79/152] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:36.386856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681275a-7ecc-49e1-a1fe-7044eaebf51f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:36.387142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681275a-7ecc-49e1-a1fe-7044eaebf51f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:36.387512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681275a-7ecc-49e1-a1fe-7044eaebf51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:36.414924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a681275a-7ecc-49e1-a1fe-7044eaebf51f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:36.416229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a681275a-7ecc-49e1-a1fe-7044eaebf51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:36.436121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 74/153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:01:36 2022] PUT /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51/users/a32fb42b027d40c08da7b59f1d60dc77/roles/6014fae253be445d9675cf76770c7b4c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:36.436995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07980e21-700a-4cfe-8024-573d7e52de23 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:36.509923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:36.522518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 80/154] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:36.583894 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681275a-7ecc-49e1-a1fe-7044eaebf51f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:36.605320 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07980e21-700a-4cfe-8024-573d7e52de23 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:36.663338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e4b54-58bd-430a-b37b-871a1115c1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:36.664158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e4b54-58bd-430a-b37b-871a1115c1eb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:36.664444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e4b54-58bd-430a-b37b-871a1115c1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:36.664721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e4b54-58bd-430a-b37b-871a1115c1eb None admin] PATH_INFO: `/v3/users/d695ee7d689a4ca4ae7d161f5773ca60` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:36.666038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e4b54-58bd-430a-b37b-871a1115c1eb None admin] RBAC: Authorizing `identity:get_user(user_id=d695ee7d689a4ca4ae7d161f5773ca60)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:36.667848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e4b54-58bd-430a-b37b-871a1115c1eb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:36.669680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 75/155] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:36 2022] GET /identity/v3/users/d695ee7d689a4ca4ae7d161f5773ca60 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:36.675281 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e4b54-58bd-430a-b37b-871a1115c1eb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:36.679475 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:36.745680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681275a-7ecc-49e1-a1fe-7044eaebf51f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:36.846963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e4b54-58bd-430a-b37b-871a1115c1eb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:36.870968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:36.879205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84abd8-4462-4b6a-84d4-026c03034db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:36.880034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84abd8-4462-4b6a-84d4-026c03034db2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:36.884307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84abd8-4462-4b6a-84d4-026c03034db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:36.884307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84abd8-4462-4b6a-84d4-026c03034db2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:36.896731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84abd8-4462-4b6a-84d4-026c03034db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:36.901504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167a0ca-214a-4e2c-be69-3ea05c1d09de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:36.902169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a0ca-214a-4e2c-be69-3ea05c1d09de None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:36.902523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a0ca-214a-4e2c-be69-3ea05c1d09de None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:36.902833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167a0ca-214a-4e2c-be69-3ea05c1d09de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ea84abd8-4462-4b6a-84d4-026c03034db2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:36.910940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:36.911841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:36.912588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 81/156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:01:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:36.918466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84abd8-4462-4b6a-84d4-026c03034db2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:36.995205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167a0ca-214a-4e2c-be69-3ea05c1d09de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:36.996705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167a0ca-214a-4e2c-be69-3ea05c1d09de None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:37.146465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 76/157] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 12:01:37.147928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d312e-71e4-48da-81b5-8c8c089b2663 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.148227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d312e-71e4-48da-81b5-8c8c089b2663 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.148506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d312e-71e4-48da-81b5-8c8c089b2663 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.149810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 77/158] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:37.157007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef610d1-f1fb-43cb-8839-dedf2979d2e7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.157486 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef610d1-f1fb-43cb-8839-dedf2979d2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.157999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef610d1-f1fb-43cb-8839-dedf2979d2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.173810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:37.187225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5ef610d1-f1fb-43cb-8839-dedf2979d2e7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:37.188473 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5ef610d1-f1fb-43cb-8839-dedf2979d2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:37.252485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84abd8-4462-4b6a-84d4-026c03034db2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:37.303808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 78/159] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:37.304767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef610d1-f1fb-43cb-8839-dedf2979d2e7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:37.309347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:37.376856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29584b3-38d9-4833-aa75-8ebf80380400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:37.377550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29584b3-38d9-4833-aa75-8ebf80380400 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.377838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29584b3-38d9-4833-aa75-8ebf80380400 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.378118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29584b3-38d9-4833-aa75-8ebf80380400 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.378915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29584b3-38d9-4833-aa75-8ebf80380400 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:37.379228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29584b3-38d9-4833-aa75-8ebf80380400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:37.380629 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29584b3-38d9-4833-aa75-8ebf80380400 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:37.408797 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 82/160] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 491 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:37.409685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29584b3-38d9-4833-aa75-8ebf80380400 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:37.444330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef610d1-f1fb-43cb-8839-dedf2979d2e7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:37.511239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c16908-1581-4714-a5ec-08a29cbf1c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:37.512140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c16908-1581-4714-a5ec-08a29cbf1c0c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.512430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c16908-1581-4714-a5ec-08a29cbf1c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.512711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c16908-1581-4714-a5ec-08a29cbf1c0c None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.513997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c16908-1581-4714-a5ec-08a29cbf1c0c None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:37.515589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c16908-1581-4714-a5ec-08a29cbf1c0c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:37.517950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 79/161] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:37 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:37.518594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c16908-1581-4714-a5ec-08a29cbf1c0c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:37.522606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:37.682231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29584b3-38d9-4833-aa75-8ebf80380400 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:37.683296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c16908-1581-4714-a5ec-08a29cbf1c0c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:37.743551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b65aaf-3dcb-460a-ac0c-851e354ba719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:37.745660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b65aaf-3dcb-460a-ac0c-851e354ba719 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.746678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b65aaf-3dcb-460a-ac0c-851e354ba719 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.747613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b65aaf-3dcb-460a-ac0c-851e354ba719 None admin] PATH_INFO: `/v3/users/87e06b5266a1425fb48c02407f951f73` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.750770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b65aaf-3dcb-460a-ac0c-851e354ba719 None admin] RBAC: Authorizing `identity:get_user(user_id=87e06b5266a1425fb48c02407f951f73)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:37.751596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95d717-4cd5-46e9-9c40-e6314a1a313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:37.753037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95d717-4cd5-46e9-9c40-e6314a1a313e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.754502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b65aaf-3dcb-460a-ac0c-851e354ba719 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:37.754992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95d717-4cd5-46e9-9c40-e6314a1a313e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.755548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95d717-4cd5-46e9-9c40-e6314a1a313e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.759639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 80/162] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:37 2022] GET /identity/v3/users/87e06b5266a1425fb48c02407f951f73 => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:37.760633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b65aaf-3dcb-460a-ac0c-851e354ba719 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:37.782947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95d717-4cd5-46e9-9c40-e6314a1a313e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ad95d717-4cd5-46e9-9c40-e6314a1a313e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:37.793710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:37.794659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:37.795425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 83/163] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:01:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:37.796813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab2262-78be-46ff-bd8f-1959be459275 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:37.796980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab2262-78be-46ff-bd8f-1959be459275 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:37.797222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab2262-78be-46ff-bd8f-1959be459275 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:37.799869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 84/164] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:37.799869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab2262-78be-46ff-bd8f-1959be459275 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:37.913484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:37.914276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:37.980527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b65aaf-3dcb-460a-ac0c-851e354ba719 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.039094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab2262-78be-46ff-bd8f-1959be459275 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.087156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf4e3f-200f-4389-9d2b-b5e765113aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.087835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf4e3f-200f-4389-9d2b-b5e765113aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.088130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf4e3f-200f-4389-9d2b-b5e765113aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.088415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf4e3f-200f-4389-9d2b-b5e765113aa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.089270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf4e3f-200f-4389-9d2b-b5e765113aa0 None admin] RBAC: Adding query filter params (role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=adcc2a3589e7498db55bb58c9719de51, user.id=d695ee7d689a4ca4ae7d161f5773ca60) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:38.089591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf4e3f-200f-4389-9d2b-b5e765113aa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.091110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf4e3f-200f-4389-9d2b-b5e765113aa0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.094479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e8e0b-7879-4d9f-baaf-d3e1b9c3ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.095144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e8e0b-7879-4d9f-baaf-d3e1b9c3ba1b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.095429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e8e0b-7879-4d9f-baaf-d3e1b9c3ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.095705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e8e0b-7879-4d9f-baaf-d3e1b9c3ba1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.106611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 81/165] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:37 2022] GET /identity/v3/role_assignments?user.id=d695ee7d689a4ca4ae7d161f5773ca60&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:38.108218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e8e0-0788-4370-b850-5cf890638118 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.108516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e8e0-0788-4370-b850-5cf890638118 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.108798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52e8e0-0788-4370-b850-5cf890638118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.147394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5a52e8e0-0788-4370-b850-5cf890638118 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:38.148610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5a52e8e0-0788-4370-b850-5cf890638118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:38.235368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e8e0b-7879-4d9f-baaf-d3e1b9c3ba1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.237078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e8e0b-7879-4d9f-baaf-d3e1b9c3ba1b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.271989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 82/166] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:38.272952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e8e0-0788-4370-b850-5cf890638118 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.348569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:38.378773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:38.425703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52e8e0-0788-4370-b850-5cf890638118 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.477882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab5735-385b-4743-a7b8-2ce04a3b84a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.478607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab5735-385b-4743-a7b8-2ce04a3b84a3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.478903 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab5735-385b-4743-a7b8-2ce04a3b84a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.479180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab5735-385b-4743-a7b8-2ce04a3b84a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.479950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab5735-385b-4743-a7b8-2ce04a3b84a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:38.480262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab5735-385b-4743-a7b8-2ce04a3b84a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.481680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab5735-385b-4743-a7b8-2ce04a3b84a3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.491318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 83/167] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:38.496720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab5735-385b-4743-a7b8-2ce04a3b84a3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.521245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 85/168] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:38.521950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e8e0b-7879-4d9f-baaf-d3e1b9c3ba1b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.669156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab5735-385b-4743-a7b8-2ce04a3b84a3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.710852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:38.712639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e8e0b-7879-4d9f-baaf-d3e1b9c3ba1b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.724590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc073ec-6ba0-407e-b9d2-19e952bd7617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.725420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc073ec-6ba0-407e-b9d2-19e952bd7617 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.725707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc073ec-6ba0-407e-b9d2-19e952bd7617 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.726768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc073ec-6ba0-407e-b9d2-19e952bd7617 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.747488 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc073ec-6ba0-407e-b9d2-19e952bd7617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3bc073ec-6ba0-407e-b9d2-19e952bd7617 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:38.761668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:38.762650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:38.763412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 84/169] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:38.764584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc073ec-6ba0-407e-b9d2-19e952bd7617 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.774609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352201d3-78b3-414a-a4d6-f77c821be4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.775503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352201d3-78b3-414a-a4d6-f77c821be4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.775784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352201d3-78b3-414a-a4d6-f77c821be4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.776055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352201d3-78b3-414a-a4d6-f77c821be4d1 None admin] PATH_INFO: `/v3/users/d695ee7d689a4ca4ae7d161f5773ca60` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.777264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352201d3-78b3-414a-a4d6-f77c821be4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=d695ee7d689a4ca4ae7d161f5773ca60)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.778735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352201d3-78b3-414a-a4d6-f77c821be4d1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.780532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 86/170] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/users/d695ee7d689a4ca4ae7d161f5773ca60 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:38.781183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352201d3-78b3-414a-a4d6-f77c821be4d1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:38.814046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:38.913912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc073ec-6ba0-407e-b9d2-19e952bd7617 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.966619 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb44a35-6c81-48cb-96e4-71da266f630e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:38.967427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44a35-6c81-48cb-96e4-71da266f630e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:38.967712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44a35-6c81-48cb-96e4-71da266f630e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:38.967992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44a35-6c81-48cb-96e4-71da266f630e None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:38.972789 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352201d3-78b3-414a-a4d6-f77c821be4d1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:38.975330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb44a35-6c81-48cb-96e4-71da266f630e None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:38.977217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb44a35-6c81-48cb-96e4-71da266f630e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:38.980400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 85/171] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:38.986956 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb44a35-6c81-48cb-96e4-71da266f630e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:39.051984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7e111-b36a-4e71-a663-ff00808f1039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:39.052755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7e111-b36a-4e71-a663-ff00808f1039 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.053036 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7e111-b36a-4e71-a663-ff00808f1039 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.053309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7e111-b36a-4e71-a663-ff00808f1039 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.054129 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7e111-b36a-4e71-a663-ff00808f1039 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:39.054461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7e111-b36a-4e71-a663-ff00808f1039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:39.055904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7e111-b36a-4e71-a663-ff00808f1039 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.069133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 87/172] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:39.070018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7e111-b36a-4e71-a663-ff00808f1039 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:39.082327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:39.196747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb44a35-6c81-48cb-96e4-71da266f630e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:39.259815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:39.309047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7320f0b-cbe4-407b-a90b-841c1e2cfb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:39.309047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7320f0b-cbe4-407b-a90b-841c1e2cfb27 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.309047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7320f0b-cbe4-407b-a90b-841c1e2cfb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.309047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7320f0b-cbe4-407b-a90b-841c1e2cfb27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.346148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7e111-b36a-4e71-a663-ff00808f1039 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:39.422911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac8adc-feb2-4678-b257-68ee48593018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:39.423807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac8adc-feb2-4678-b257-68ee48593018 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.423930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac8adc-feb2-4678-b257-68ee48593018 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.424271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac8adc-feb2-4678-b257-68ee48593018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.425012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac8adc-feb2-4678-b257-68ee48593018 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=adcc2a3589e7498db55bb58c9719de51, user.id=87e06b5266a1425fb48c02407f951f73) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:39.425293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac8adc-feb2-4678-b257-68ee48593018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:39.426853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac8adc-feb2-4678-b257-68ee48593018 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.436022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 88/173] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:39 2022] GET /identity/v3/role_assignments?user.id=87e06b5266a1425fb48c02407f951f73&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=bd295cb5d1194b469beff0381c6b23af => generated 608 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:39.436690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac8adc-feb2-4678-b257-68ee48593018 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:39.501769 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7320f0b-cbe4-407b-a90b-841c1e2cfb27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:39.503372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7320f0b-cbe4-407b-a90b-841c1e2cfb27 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.699920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:39.721984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac8adc-feb2-4678-b257-68ee48593018 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:39.799648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd101c6d-da3f-4fcb-a490-93dfa5f9e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:39.801479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd101c6d-da3f-4fcb-a490-93dfa5f9e6aa None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.801773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd101c6d-da3f-4fcb-a490-93dfa5f9e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.803057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd101c6d-da3f-4fcb-a490-93dfa5f9e6aa None admin] PATH_INFO: `/v3/users/d695ee7d689a4ca4ae7d161f5773ca60` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.803638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd101c6d-da3f-4fcb-a490-93dfa5f9e6aa None admin] RBAC: Authorizing `identity:get_user(user_id=d695ee7d689a4ca4ae7d161f5773ca60)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:39.805960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd101c6d-da3f-4fcb-a490-93dfa5f9e6aa None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:39.808705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 89/174] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:39 2022] GET /identity/v3/users/d695ee7d689a4ca4ae7d161f5773ca60 => generated 275 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:39.809958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4fe52-be1b-4479-884c-58eed0d9ea71 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.810255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4fe52-be1b-4479-884c-58eed0d9ea71 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.810582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4fe52-be1b-4479-884c-58eed0d9ea71 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.811878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 90/175] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:39.813114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da80b0-2cab-4973-86e1-699bbcf3b517 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.813468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da80b0-2cab-4973-86e1-699bbcf3b517 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.813761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da80b0-2cab-4973-86e1-699bbcf3b517 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.813949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 86/176] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 833 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:39.815094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 91/177] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:39.816155 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7320f0b-cbe4-407b-a90b-841c1e2cfb27 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:39.821896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef4719-1302-4881-80d1-ede9068e183e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.822161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef4719-1302-4881-80d1-ede9068e183e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.822467 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef4719-1302-4881-80d1-ede9068e183e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.835901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:39.850847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-92ef4719-1302-4881-80d1-ede9068e183e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:39.852675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-92ef4719-1302-4881-80d1-ede9068e183e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:39.949673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 92/178] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:39.952373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafc64a-561c-484e-91f8-568db823736f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:39.953274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafc64a-561c-484e-91f8-568db823736f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:39.953576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafc64a-561c-484e-91f8-568db823736f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:39.966029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7320f0b-cbe4-407b-a90b-841c1e2cfb27 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:39.977599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8bafc64a-561c-484e-91f8-568db823736f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:39.979838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8bafc64a-561c-484e-91f8-568db823736f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:40.016079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7986f8-124d-410a-90a7-682159eac540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:40.016854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7986f8-124d-410a-90a7-682159eac540 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.017219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7986f8-124d-410a-90a7-682159eac540 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.017564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7986f8-124d-410a-90a7-682159eac540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.054377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:40.084770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 93/179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:40.084770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafc64a-561c-484e-91f8-568db823736f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:40.137448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7986f8-124d-410a-90a7-682159eac540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:40.140645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7986f8-124d-410a-90a7-682159eac540 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:40.157783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:40.201325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafc64a-561c-484e-91f8-568db823736f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:40.240468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016b826-c950-45e7-8cca-f6297e7d06d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:40.241372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016b826-c950-45e7-8cca-f6297e7d06d8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.241662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016b826-c950-45e7-8cca-f6297e7d06d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.241941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016b826-c950-45e7-8cca-f6297e7d06d8 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.243295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016b826-c950-45e7-8cca-f6297e7d06d8 None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:40.244773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016b826-c950-45e7-8cca-f6297e7d06d8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:40.251882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 94/180] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:40 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:40.253252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf16e4-482d-47a4-96a7-6c1fd81c8436 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.253549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf16e4-482d-47a4-96a7-6c1fd81c8436 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.253833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf16e4-482d-47a4-96a7-6c1fd81c8436 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.255367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 95/181] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:40.256027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf16e4-482d-47a4-96a7-6c1fd81c8436 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:40.278614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 87/182] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 12:01:40.284016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7986f8-124d-410a-90a7-682159eac540 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:40.347574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:40.374116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf16e4-482d-47a4-96a7-6c1fd81c8436 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:40.394259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7986f8-124d-410a-90a7-682159eac540 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:40.406570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:40.411403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ba7e8-6f73-476c-bf3e-1b37758b1197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:40.412206 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ba7e8-6f73-476c-bf3e-1b37758b1197 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.412695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ba7e8-6f73-476c-bf3e-1b37758b1197 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.412828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ba7e8-6f73-476c-bf3e-1b37758b1197 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.413920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ba7e8-6f73-476c-bf3e-1b37758b1197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:40.415455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ba7e8-6f73-476c-bf3e-1b37758b1197 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:40.417068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 96/183] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:40.422688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ba7e8-6f73-476c-bf3e-1b37758b1197 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:40.429752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2325fd-9e14-4f7f-9920-9457638fe9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:40.430625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2325fd-9e14-4f7f-9920-9457638fe9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.430914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2325fd-9e14-4f7f-9920-9457638fe9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.431197 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2325fd-9e14-4f7f-9920-9457638fe9d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.442522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2325fd-9e14-4f7f-9920-9457638fe9d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0b2325fd-9e14-4f7f-9920-9457638fe9d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:40.450284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:40.451175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:40.452250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 88/184] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:01:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:40.454190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d010556-5abc-483f-9e6f-85523dce49e6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.454190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d010556-5abc-483f-9e6f-85523dce49e6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.454190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d010556-5abc-483f-9e6f-85523dce49e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.470697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0d010556-5abc-483f-9e6f-85523dce49e6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:40.471923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0d010556-5abc-483f-9e6f-85523dce49e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:40.528064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ba7e8-6f73-476c-bf3e-1b37758b1197 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:40.540853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 89/185] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:40.541703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d010556-5abc-483f-9e6f-85523dce49e6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:40.578739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcce53-c549-464f-8cb6-3de3b1f56c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:40.579835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcce53-c549-464f-8cb6-3de3b1f56c63 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.580125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcce53-c549-464f-8cb6-3de3b1f56c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.586691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcce53-c549-464f-8cb6-3de3b1f56c63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.586691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcce53-c549-464f-8cb6-3de3b1f56c63 None admin] RBAC: Adding query filter params (role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=adcc2a3589e7498db55bb58c9719de51, user.id=d695ee7d689a4ca4ae7d161f5773ca60) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:40.586691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcce53-c549-464f-8cb6-3de3b1f56c63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:40.586691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcce53-c549-464f-8cb6-3de3b1f56c63 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:40.589456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 97/186] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:40 2022] GET /identity/v3/role_assignments?user.id=d695ee7d689a4ca4ae7d161f5773ca60&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:40.590842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12452c0-43d3-42d9-ba00-7992de6b7f3d None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.591142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12452c0-43d3-42d9-ba00-7992de6b7f3d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.591424 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12452c0-43d3-42d9-ba00-7992de6b7f3d None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.592412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:40.592682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 98/187] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:40.593440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12452c0-43d3-42d9-ba00-7992de6b7f3d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:40.656333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d010556-5abc-483f-9e6f-85523dce49e6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:40.676891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:40.697224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e045-c290-4fdf-b7ca-eda02a02829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:40.698119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e045-c290-4fdf-b7ca-eda02a02829f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.698429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e045-c290-4fdf-b7ca-eda02a02829f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.698726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e045-c290-4fdf-b7ca-eda02a02829f None admin] PATH_INFO: `/v3/projects/e0f2ea73517b4bb985689b50342c2cbe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.700032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206e045-c290-4fdf-b7ca-eda02a02829f None admin] RBAC: Authorizing `identity:get_project(project_id=e0f2ea73517b4bb985689b50342c2cbe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:40.701513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206e045-c290-4fdf-b7ca-eda02a02829f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:40.703385 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 90/188] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:40 2022] GET /identity/v3/projects/e0f2ea73517b4bb985689b50342c2cbe => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:40.704211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e045-c290-4fdf-b7ca-eda02a02829f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:40.727230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12452c0-43d3-42d9-ba00-7992de6b7f3d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:40.770911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431f272-275d-46ea-af30-714455c63864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:40.771606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f272-275d-46ea-af30-714455c63864 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.771897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f272-275d-46ea-af30-714455c63864 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.772175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f272-275d-46ea-af30-714455c63864 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.773032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431f272-275d-46ea-af30-714455c63864 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:40.774664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431f272-275d-46ea-af30-714455c63864 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:40.821342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e045-c290-4fdf-b7ca-eda02a02829f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:40.837103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:40.858511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb2e72-907a-48a7-8ec2-b4c4ce124e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:40.859913 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb2e72-907a-48a7-8ec2-b4c4ce124e6c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.860346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb2e72-907a-48a7-8ec2-b4c4ce124e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.860626 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb2e72-907a-48a7-8ec2-b4c4ce124e6c None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.861388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb2e72-907a-48a7-8ec2-b4c4ce124e6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:40.861704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb2e72-907a-48a7-8ec2-b4c4ce124e6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:40.863115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb2e72-907a-48a7-8ec2-b4c4ce124e6c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:40.871581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 91/189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:40.872471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb2e72-907a-48a7-8ec2-b4c4ce124e6c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:40.949896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 99/190] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:01:40 2022] POST /identity/v3/users => generated 297 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:40.949896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750f01c-c5ed-487c-8936-3d2f88b00a77 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:40.949896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750f01c-c5ed-487c-8936-3d2f88b00a77 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:40.949896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750f01c-c5ed-487c-8936-3d2f88b00a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:40.968817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0750f01c-c5ed-487c-8936-3d2f88b00a77 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:40.970107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0750f01c-c5ed-487c-8936-3d2f88b00a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:40.987841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb2e72-907a-48a7-8ec2-b4c4ce124e6c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:41.035118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d10a8-15b9-4ce1-80f2-18c535dedf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:41.036252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d10a8-15b9-4ce1-80f2-18c535dedf6d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:41.036687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d10a8-15b9-4ce1-80f2-18c535dedf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:41.037117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d10a8-15b9-4ce1-80f2-18c535dedf6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:41.050882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d10a8-15b9-4ce1-80f2-18c535dedf6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:41.053145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 100/191] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:41.054007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750f01c-c5ed-487c-8936-3d2f88b00a77 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-589d10a8-15b9-4ce1-80f2-18c535dedf6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:41.060325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:41.061269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:41.063011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 92/192] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:01:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:41.063663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d10a8-15b9-4ce1-80f2-18c535dedf6d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:41.110707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:41.123952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:41.191766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750f01c-c5ed-487c-8936-3d2f88b00a77 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:41.193997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d10a8-15b9-4ce1-80f2-18c535dedf6d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:41.239015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1615a15-8b66-4bd1-9ad9-198feccc358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:41.239690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1615a15-8b66-4bd1-9ad9-198feccc358b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:41.239979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1615a15-8b66-4bd1-9ad9-198feccc358b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:41.240137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51108d8e-1903-4382-addf-56dd16183cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:41.240323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1615a15-8b66-4bd1-9ad9-198feccc358b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:41.240985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51108d8e-1903-4382-addf-56dd16183cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:41.241180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1615a15-8b66-4bd1-9ad9-198feccc358b None admin] RBAC: Adding query filter params (role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=e0f2ea73517b4bb985689b50342c2cbe, user.id=d695ee7d689a4ca4ae7d161f5773ca60) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:41.241306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51108d8e-1903-4382-addf-56dd16183cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:41.241498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1615a15-8b66-4bd1-9ad9-198feccc358b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:41.241595 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51108d8e-1903-4382-addf-56dd16183cf9 None admin] PATH_INFO: `/v3/users/16c372374bf74737995c2046fa929fc7` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:41.243155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1615a15-8b66-4bd1-9ad9-198feccc358b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:41.254614 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 101/193] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:41 2022] GET /identity/v3/role_assignments?user.id=d695ee7d689a4ca4ae7d161f5773ca60&scope.project.id=e0f2ea73517b4bb985689b50342c2cbe&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:41.256597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1615a15-8b66-4bd1-9ad9-198feccc358b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:41.258261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51108d8e-1903-4382-addf-56dd16183cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=16c372374bf74737995c2046fa929fc7)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:41.260033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51108d8e-1903-4382-addf-56dd16183cf9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:41.262707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 93/194] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:41 2022] GET /identity/v3/users/16c372374bf74737995c2046fa929fc7 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:41.263927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51108d8e-1903-4382-addf-56dd16183cf9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:41.399434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:41.407410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51108d8e-1903-4382-addf-56dd16183cf9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:41.425332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1615a15-8b66-4bd1-9ad9-198feccc358b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:41.448516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:41.449070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310accc-f55d-478b-8526-d684fba9f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:41.450383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310accc-f55d-478b-8526-d684fba9f073 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:41.450699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310accc-f55d-478b-8526-d684fba9f073 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:41.450984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310accc-f55d-478b-8526-d684fba9f073 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:41.476371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310accc-f55d-478b-8526-d684fba9f073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:41.479460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fe10d-081e-4fcb-aeff-03338c752df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:41.480144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fe10d-081e-4fcb-aeff-03338c752df1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:41.480435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fe10d-081e-4fcb-aeff-03338c752df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:41.480715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fe10d-081e-4fcb-aeff-03338c752df1 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:41.481469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fe10d-081e-4fcb-aeff-03338c752df1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:41.481788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fe10d-081e-4fcb-aeff-03338c752df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:41.483294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fe10d-081e-4fcb-aeff-03338c752df1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4310accc-f55d-478b-8526-d684fba9f073 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:41.489407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:41.490328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:41.491153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:41.491750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 94/195] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:41.492552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310accc-f55d-478b-8526-d684fba9f073 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:41.495370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 102/196] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:01:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:41.498537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fe10d-081e-4fcb-aeff-03338c752df1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:41.624845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310accc-f55d-478b-8526-d684fba9f073 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:41.686096 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e856ef-3c28-463b-adff-fb2f08aed244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:41.687684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e856ef-3c28-463b-adff-fb2f08aed244 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:41.688177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e856ef-3c28-463b-adff-fb2f08aed244 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:41.688671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e856ef-3c28-463b-adff-fb2f08aed244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:41.695401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fe10d-081e-4fcb-aeff-03338c752df1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:41.717451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:41.741770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36345085-dedb-4d09-8de6-9149dce1aa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:41.743180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36345085-dedb-4d09-8de6-9149dce1aa83 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:41.743673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36345085-dedb-4d09-8de6-9149dce1aa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:41.744498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36345085-dedb-4d09-8de6-9149dce1aa83 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:41.745641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36345085-dedb-4d09-8de6-9149dce1aa83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:41.746299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36345085-dedb-4d09-8de6-9149dce1aa83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:41.748111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36345085-dedb-4d09-8de6-9149dce1aa83 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:41.761020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 103/197] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:41.765741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36345085-dedb-4d09-8de6-9149dce1aa83 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:41.778547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e856ef-3c28-463b-adff-fb2f08aed244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:41.781633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e856ef-3c28-463b-adff-fb2f08aed244 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:41.812744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:41.880810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36345085-dedb-4d09-8de6-9149dce1aa83 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:41.893894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 95/198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:41.895193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e856ef-3c28-463b-adff-fb2f08aed244 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:41.918932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c942c-c019-43ad-8d45-3fcf1d9b13a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:41.919818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c942c-c019-43ad-8d45-3fcf1d9b13a8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:41.920105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c942c-c019-43ad-8d45-3fcf1d9b13a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:41.920409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c942c-c019-43ad-8d45-3fcf1d9b13a8 None admin] PATH_INFO: `/v3/users/16c372374bf74737995c2046fa929fc7` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:41.921873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c942c-c019-43ad-8d45-3fcf1d9b13a8 None admin] RBAC: Authorizing `identity:get_user(user_id=16c372374bf74737995c2046fa929fc7)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:41.923556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c942c-c019-43ad-8d45-3fcf1d9b13a8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:41.925122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 104/199] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:41 2022] GET /identity/v3/users/16c372374bf74737995c2046fa929fc7 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:41.930352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c942c-c019-43ad-8d45-3fcf1d9b13a8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:41.958678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.005391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e856ef-3c28-463b-adff-fb2f08aed244 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.044799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c942c-c019-43ad-8d45-3fcf1d9b13a8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.053148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.053710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d056267-c2e5-49cd-833f-a78956165d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.054861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d056267-c2e5-49cd-833f-a78956165d31 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.055353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d056267-c2e5-49cd-833f-a78956165d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.055852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d056267-c2e5-49cd-833f-a78956165d31 None admin] PATH_INFO: `/v3/users/a32fb42b027d40c08da7b59f1d60dc77` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.057290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d056267-c2e5-49cd-833f-a78956165d31 None admin] RBAC: Authorizing `identity:get_user(user_id=a32fb42b027d40c08da7b59f1d60dc77)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.059014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d056267-c2e5-49cd-833f-a78956165d31 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.060919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 96/200] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:41 2022] GET /identity/v3/users/a32fb42b027d40c08da7b59f1d60dc77 => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:42.061979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d056267-c2e5-49cd-833f-a78956165d31 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.088570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6ec5c-fcd9-4259-9f17-71a13aca3819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.089450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6ec5c-fcd9-4259-9f17-71a13aca3819 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.089732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6ec5c-fcd9-4259-9f17-71a13aca3819 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.090281 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6ec5c-fcd9-4259-9f17-71a13aca3819 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.091238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6ec5c-fcd9-4259-9f17-71a13aca3819 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.092694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6ec5c-fcd9-4259-9f17-71a13aca3819 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.094574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 105/201] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:41 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:42.095493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6ec5c-fcd9-4259-9f17-71a13aca3819 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.175928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d056267-c2e5-49cd-833f-a78956165d31 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.200477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.214511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6ec5c-fcd9-4259-9f17-71a13aca3819 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.214717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e5d4b-8d14-4fc2-bba5-6f43d77fd78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.214717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e5d4b-8d14-4fc2-bba5-6f43d77fd78f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.214717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e5d4b-8d14-4fc2-bba5-6f43d77fd78f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.214717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e5d4b-8d14-4fc2-bba5-6f43d77fd78f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.246839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b1597-81cd-44c4-9ab4-0ef511721e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.247492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b1597-81cd-44c4-9ab4-0ef511721e61 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.247776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b1597-81cd-44c4-9ab4-0ef511721e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.248054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b1597-81cd-44c4-9ab4-0ef511721e61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.288686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e5d4b-8d14-4fc2-bba5-6f43d77fd78f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.290548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e5d4b-8d14-4fc2-bba5-6f43d77fd78f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.294758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.323032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b1597-81cd-44c4-9ab4-0ef511721e61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.324518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b1597-81cd-44c4-9ab4-0ef511721e61 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.403297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 97/202] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:42.404523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e5d4b-8d14-4fc2-bba5-6f43d77fd78f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.435338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.447589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 106/203] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:42.448841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b1597-81cd-44c4-9ab4-0ef511721e61 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.518689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e5d4b-8d14-4fc2-bba5-6f43d77fd78f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.540133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.557380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887eb7e-9dd0-4c39-9668-b47985a30f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.558447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887eb7e-9dd0-4c39-9668-b47985a30f61 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.558955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887eb7e-9dd0-4c39-9668-b47985a30f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.559443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887eb7e-9dd0-4c39-9668-b47985a30f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.560482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887eb7e-9dd0-4c39-9668-b47985a30f61 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=f00dbca6b5e149df98838a84028da079, user.id=16c372374bf74737995c2046fa929fc7) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:42.561002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887eb7e-9dd0-4c39-9668-b47985a30f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.562889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887eb7e-9dd0-4c39-9668-b47985a30f61 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.568807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b1597-81cd-44c4-9ab4-0ef511721e61 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.569428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 98/204] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/role_assignments?user.id=16c372374bf74737995c2046fa929fc7&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=bd295cb5d1194b469beff0381c6b23af => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:42.570476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887eb7e-9dd0-4c39-9668-b47985a30f61 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.612905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad011f8-85db-4355-8953-507aaeb2c3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.613810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad011f8-85db-4355-8953-507aaeb2c3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.614097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad011f8-85db-4355-8953-507aaeb2c3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.614376 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad011f8-85db-4355-8953-507aaeb2c3c3 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.615816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad011f8-85db-4355-8953-507aaeb2c3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.617331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad011f8-85db-4355-8953-507aaeb2c3c3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.619261 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 107/205] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:42.625794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad011f8-85db-4355-8953-507aaeb2c3c3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.669235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:42.687587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887eb7e-9dd0-4c39-9668-b47985a30f61 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.725794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1755e7c-c3fe-4180-b96d-38e2fbfb36a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.727118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1755e7c-c3fe-4180-b96d-38e2fbfb36a7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.727614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1755e7c-c3fe-4180-b96d-38e2fbfb36a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.728274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1755e7c-c3fe-4180-b96d-38e2fbfb36a7 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.730167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1755e7c-c3fe-4180-b96d-38e2fbfb36a7 None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.731983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1755e7c-c3fe-4180-b96d-38e2fbfb36a7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.734606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 99/206] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:42.742135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1755e7c-c3fe-4180-b96d-38e2fbfb36a7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:42.753853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad011f8-85db-4355-8953-507aaeb2c3c3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.843653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f391ed-3e6f-4844-9043-4b57a71b6328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.844822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f391ed-3e6f-4844-9043-4b57a71b6328 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.845450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f391ed-3e6f-4844-9043-4b57a71b6328 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.846121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f391ed-3e6f-4844-9043-4b57a71b6328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.847233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f391ed-3e6f-4844-9043-4b57a71b6328 None admin] RBAC: Adding query filter params (role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=f00dbca6b5e149df98838a84028da079, user.id=16c372374bf74737995c2046fa929fc7) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:42.849147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f391ed-3e6f-4844-9043-4b57a71b6328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.851183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f391ed-3e6f-4844-9043-4b57a71b6328 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.864364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 108/207] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/role_assignments?user.id=16c372374bf74737995c2046fa929fc7&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:42.930772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1755e7c-c3fe-4180-b96d-38e2fbfb36a7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:42.973671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd01ce-5b7e-47fe-bd7c-17b9f40c3ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:42.974355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd01ce-5b7e-47fe-bd7c-17b9f40c3ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:42.974674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd01ce-5b7e-47fe-bd7c-17b9f40c3ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:42.974957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd01ce-5b7e-47fe-bd7c-17b9f40c3ff3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:42.975816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd01ce-5b7e-47fe-bd7c-17b9f40c3ff3 None admin] RBAC: Adding query filter params (role.id=6014fae253be445d9675cf76770c7b4c, scope.project.id=adcc2a3589e7498db55bb58c9719de51, user.id=a32fb42b027d40c08da7b59f1d60dc77) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:42.976138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd01ce-5b7e-47fe-bd7c-17b9f40c3ff3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:42.977628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd01ce-5b7e-47fe-bd7c-17b9f40c3ff3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:42.985827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 100/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:42 2022] GET /identity/v3/role_assignments?user.id=a32fb42b027d40c08da7b59f1d60dc77&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=6014fae253be445d9675cf76770c7b4c => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:43.641302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2ea0-6ed7-4de3-8fbe-c8867d646288 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.641996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2ea0-6ed7-4de3-8fbe-c8867d646288 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.642519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2ea0-6ed7-4de3-8fbe-c8867d646288 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.644159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 109/209] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:43.650033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef31dc-3381-4fe9-9792-f6ac999d8163 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.650620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef31dc-3381-4fe9-9792-f6ac999d8163 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.651112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef31dc-3381-4fe9-9792-f6ac999d8163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.674085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-92ef31dc-3381-4fe9-9792-f6ac999d8163 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:43.676250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-92ef31dc-3381-4fe9-9792-f6ac999d8163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:43.700172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:43.760983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 101/210] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:43.784917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec2ea0-6ed7-4de3-8fbe-c8867d646288 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:43.795395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:43.878708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacf7e4-af10-4277-ab14-1aeab7d06647 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.879985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacf7e4-af10-4277-ab14-1aeab7d06647 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.880766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eacf7e4-af10-4277-ab14-1aeab7d06647 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.883003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 102/211] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:43.892648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b32ec-a8b6-4843-bf1e-75ffc0dcfdd9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.893264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b32ec-a8b6-4843-bf1e-75ffc0dcfdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.893766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b32ec-a8b6-4843-bf1e-75ffc0dcfdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.911001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec2ea0-6ed7-4de3-8fbe-c8867d646288 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:43.918067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-630b32ec-a8b6-4843-bf1e-75ffc0dcfdd9 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:43.920201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-630b32ec-a8b6-4843-bf1e-75ffc0dcfdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:43.950214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583843b-d5ee-425f-b589-055c4bf6e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:43.951161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583843b-d5ee-425f-b589-055c4bf6e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:43.951444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583843b-d5ee-425f-b589-055c4bf6e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:43.951779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583843b-d5ee-425f-b589-055c4bf6e8f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:43.964526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583843b-d5ee-425f-b589-055c4bf6e8f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8583843b-d5ee-425f-b589-055c4bf6e8f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:43.973941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:43.974921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:43.980695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 110/212] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:43.982935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583843b-d5ee-425f-b589-055c4bf6e8f6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:44.014517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 103/213] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:44.044469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b32ec-a8b6-4843-bf1e-75ffc0dcfdd9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:44.067419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:44.112932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583843b-d5ee-425f-b589-055c4bf6e8f6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:44.150898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:44.154281 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a04099-fb45-419b-ae2a-eaf54007d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:44.155182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04099-fb45-419b-ae2a-eaf54007d643 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:44.155500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04099-fb45-419b-ae2a-eaf54007d643 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:44.155790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04099-fb45-419b-ae2a-eaf54007d643 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:44.157127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a04099-fb45-419b-ae2a-eaf54007d643 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:44.157438 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a04099-fb45-419b-ae2a-eaf54007d643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:44.159301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a04099-fb45-419b-ae2a-eaf54007d643 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:44.174496 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 111/214] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:44.177114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a04099-fb45-419b-ae2a-eaf54007d643 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:44.189423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b32ec-a8b6-4843-bf1e-75ffc0dcfdd9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:44.239241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7cee1-f193-4283-af5f-8bec13fc5dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:44.240693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7cee1-f193-4283-af5f-8bec13fc5dae None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:44.241600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7cee1-f193-4283-af5f-8bec13fc5dae None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:44.242454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7cee1-f193-4283-af5f-8bec13fc5dae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:44.244831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7cee1-f193-4283-af5f-8bec13fc5dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:44.247255 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7cee1-f193-4283-af5f-8bec13fc5dae None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:44.251014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 104/215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:44.256835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7cee1-f193-4283-af5f-8bec13fc5dae None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:44.294227 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a04099-fb45-419b-ae2a-eaf54007d643 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:44.326442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:44.335040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe3219-81da-4c20-8197-e4bb4b9e3211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:44.335937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe3219-81da-4c20-8197-e4bb4b9e3211 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:44.336223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe3219-81da-4c20-8197-e4bb4b9e3211 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:44.342333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe3219-81da-4c20-8197-e4bb4b9e3211 None admin] PATH_INFO: `/v3/users/2b916df52426485fbd88bc259fc97953` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:44.351879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe3219-81da-4c20-8197-e4bb4b9e3211 None admin] RBAC: Authorizing `identity:get_user(user_id=2b916df52426485fbd88bc259fc97953)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:44.353414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe3219-81da-4c20-8197-e4bb4b9e3211 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:44.359745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 112/216] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:44 2022] GET /identity/v3/users/2b916df52426485fbd88bc259fc97953 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:44.361700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe3219-81da-4c20-8197-e4bb4b9e3211 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:44.399965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7cee1-f193-4283-af5f-8bec13fc5dae None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:44.438844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08f935-845d-4f59-bfdc-20e431cbc5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:44.439457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08f935-845d-4f59-bfdc-20e431cbc5bf None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:44.439749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08f935-845d-4f59-bfdc-20e431cbc5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:44.440034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08f935-845d-4f59-bfdc-20e431cbc5bf None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:44.440767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08f935-845d-4f59-bfdc-20e431cbc5bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:44.442225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08f935-845d-4f59-bfdc-20e431cbc5bf None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:44.480474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe3219-81da-4c20-8197-e4bb4b9e3211 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:44.516157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed7d50-2fdd-4653-908e-0c4a5d3791eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:44.516965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed7d50-2fdd-4653-908e-0c4a5d3791eb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:44.517352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed7d50-2fdd-4653-908e-0c4a5d3791eb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:44.517729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed7d50-2fdd-4653-908e-0c4a5d3791eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:44.581427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:44.596506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed7d50-2fdd-4653-908e-0c4a5d3791eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:44.598718 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed7d50-2fdd-4653-908e-0c4a5d3791eb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:44.612301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 105/217] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:01:44 2022] POST /identity/v3/users => generated 297 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:01:44.744383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 113/218] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 12:01:44.751229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08f935-845d-4f59-bfdc-20e431cbc5bf None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:44.754328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:44.881588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08f935-845d-4f59-bfdc-20e431cbc5bf None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:44.919053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987c6c6-9248-428c-a1fa-2c2f70776bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:44.920008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987c6c6-9248-428c-a1fa-2c2f70776bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:44.920268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987c6c6-9248-428c-a1fa-2c2f70776bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:44.920506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987c6c6-9248-428c-a1fa-2c2f70776bf5 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:44.922372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987c6c6-9248-428c-a1fa-2c2f70776bf5 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:44.924768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987c6c6-9248-428c-a1fa-2c2f70776bf5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:44.926110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 106/219] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:44 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:44.931702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed7d50-2fdd-4653-908e-0c4a5d3791eb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:45.035145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:45.041674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c26d8-3a6d-4ff4-a79e-b1fd05547c78 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:45.042020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c26d8-3a6d-4ff4-a79e-b1fd05547c78 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:45.042542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c26d8-3a6d-4ff4-a79e-b1fd05547c78 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:45.043953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 107/220] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:45.050836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bb3cb-f20c-4388-adb7-264c8a4785fe None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:45.051124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bb3cb-f20c-4388-adb7-264c8a4785fe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:45.052659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bb3cb-f20c-4388-adb7-264c8a4785fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:45.058293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed7d50-2fdd-4653-908e-0c4a5d3791eb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:45.077212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-003bb3cb-f20c-4388-adb7-264c8a4785fe None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:45.078814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-003bb3cb-f20c-4388-adb7-264c8a4785fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:45.106216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d279d-1fe5-4c02-8f2a-602a91418da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:45.106988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d279d-1fe5-4c02-8f2a-602a91418da8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:45.107271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d279d-1fe5-4c02-8f2a-602a91418da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:45.107542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d279d-1fe5-4c02-8f2a-602a91418da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:45.108383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d279d-1fe5-4c02-8f2a-602a91418da8 None admin] RBAC: Adding query filter params (role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=f00dbca6b5e149df98838a84028da079, user.id=2b916df52426485fbd88bc259fc97953) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:45.109385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d279d-1fe5-4c02-8f2a-602a91418da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:45.110990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d279d-1fe5-4c02-8f2a-602a91418da8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:45.118186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 114/221] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:44 2022] GET /identity/v3/role_assignments?user.id=2b916df52426485fbd88bc259fc97953&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:45.164357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:45.169927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 108/222] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:45.193064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d279d-1fe5-4c02-8f2a-602a91418da8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:45.363413 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d279d-1fe5-4c02-8f2a-602a91418da8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:45.402628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688b4fa-65b8-4a61-a5b1-9da559e6f45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:45.403441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688b4fa-65b8-4a61-a5b1-9da559e6f45f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:45.403724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688b4fa-65b8-4a61-a5b1-9da559e6f45f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:45.403998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688b4fa-65b8-4a61-a5b1-9da559e6f45f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:45.416186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e688b4fa-65b8-4a61-a5b1-9da559e6f45f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e688b4fa-65b8-4a61-a5b1-9da559e6f45f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:45.425905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:45.426859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:45.428097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 115/223] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:01:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:45.441251 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bb3cb-f20c-4388-adb7-264c8a4785fe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:45.489167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7529cf-1252-499d-a1d8-7d4a362a88e0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:45.489525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7529cf-1252-499d-a1d8-7d4a362a88e0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:45.489739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7529cf-1252-499d-a1d8-7d4a362a88e0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:45.491337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 116/224] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:45.496570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289d05c-e2d4-49fb-a1aa-3ddea87ed13f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:45.496849 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289d05c-e2d4-49fb-a1aa-3ddea87ed13f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:45.497137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289d05c-e2d4-49fb-a1aa-3ddea87ed13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:45.523669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3289d05c-e2d4-49fb-a1aa-3ddea87ed13f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:45.524886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3289d05c-e2d4-49fb-a1aa-3ddea87ed13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:45.547240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:45.616718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bb3cb-f20c-4388-adb7-264c8a4785fe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:45.661638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd6d4d-ccd2-4120-8ba0-5cde6fe09eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:45.662328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd6d4d-ccd2-4120-8ba0-5cde6fe09eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:45.662634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd6d4d-ccd2-4120-8ba0-5cde6fe09eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:45.662910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd6d4d-ccd2-4120-8ba0-5cde6fe09eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:45.664103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd6d4d-ccd2-4120-8ba0-5cde6fe09eb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:45.664551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd6d4d-ccd2-4120-8ba0-5cde6fe09eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:45.665972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd6d4d-ccd2-4120-8ba0-5cde6fe09eb9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:45.678070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 109/225] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:45.688230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd6d4d-ccd2-4120-8ba0-5cde6fe09eb9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:45.701677 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 117/226] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 12:01:45.739352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289d05c-e2d4-49fb-a1aa-3ddea87ed13f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:45.805035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:45.886279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd6d4d-ccd2-4120-8ba0-5cde6fe09eb9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:45.965065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289d05c-e2d4-49fb-a1aa-3ddea87ed13f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:45.970740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd3fce-29e4-4584-ac1d-74d1700c7cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:45.971638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd3fce-29e4-4584-ac1d-74d1700c7cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:45.971923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd3fce-29e4-4584-ac1d-74d1700c7cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:45.972201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd3fce-29e4-4584-ac1d-74d1700c7cd3 None admin] PATH_INFO: `/v3/users/16c372374bf74737995c2046fa929fc7` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:45.982973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd3fce-29e4-4584-ac1d-74d1700c7cd3 None admin] RBAC: Authorizing `identity:get_user(user_id=16c372374bf74737995c2046fa929fc7)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:45.985007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd3fce-29e4-4584-ac1d-74d1700c7cd3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:45.986907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 110/227] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:45 2022] GET /identity/v3/users/16c372374bf74737995c2046fa929fc7 => generated 283 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:45.995393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd3fce-29e4-4584-ac1d-74d1700c7cd3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:46.023581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6833ee7-1065-4b47-8588-587e18b52edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:46.023581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6833ee7-1065-4b47-8588-587e18b52edc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:46.023581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6833ee7-1065-4b47-8588-587e18b52edc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:46.023581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6833ee7-1065-4b47-8588-587e18b52edc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:46.044472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:46.052461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6833ee7-1065-4b47-8588-587e18b52edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f6833ee7-1065-4b47-8588-587e18b52edc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:46.067486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:46.068408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:46.069171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 118/228] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:01:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:46.079682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6833ee7-1065-4b47-8588-587e18b52edc None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:46.220843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd3fce-29e4-4584-ac1d-74d1700c7cd3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:46.268871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7a45d-73a6-4e03-8ccb-2a5ce9b56697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:46.269809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7a45d-73a6-4e03-8ccb-2a5ce9b56697 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:46.270104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7a45d-73a6-4e03-8ccb-2a5ce9b56697 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:46.270386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7a45d-73a6-4e03-8ccb-2a5ce9b56697 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:46.272189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7a45d-73a6-4e03-8ccb-2a5ce9b56697 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:46.273902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7a45d-73a6-4e03-8ccb-2a5ce9b56697 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:46.276394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 111/229] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:45 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:46.278828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:46.282529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7a45d-73a6-4e03-8ccb-2a5ce9b56697 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:46.356453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6833ee7-1065-4b47-8588-587e18b52edc None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:46.399359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7a45d-73a6-4e03-8ccb-2a5ce9b56697 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:46.439674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304716be-ad50-472c-bbd5-aeb0dc09c31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:46.440882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304716be-ad50-472c-bbd5-aeb0dc09c31a None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:46.441169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304716be-ad50-472c-bbd5-aeb0dc09c31a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:46.441642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304716be-ad50-472c-bbd5-aeb0dc09c31a None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079/users/16c372374bf74737995c2046fa929fc7/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:46.444053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304716be-ad50-472c-bbd5-aeb0dc09c31a None admin] RBAC: Authorizing `identity:create_grant(project_id=f00dbca6b5e149df98838a84028da079, user_id=16c372374bf74737995c2046fa929fc7, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:46.446160 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304716be-ad50-472c-bbd5-aeb0dc09c31a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:46.465074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c65b5-afe3-4fc8-8627-9e90ffd27c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:46.467913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c65b5-afe3-4fc8-8627-9e90ffd27c10 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:46.467913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c65b5-afe3-4fc8-8627-9e90ffd27c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:46.467913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c65b5-afe3-4fc8-8627-9e90ffd27c10 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:46.467913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c65b5-afe3-4fc8-8627-9e90ffd27c10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:46.469046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c65b5-afe3-4fc8-8627-9e90ffd27c10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:46.470698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c65b5-afe3-4fc8-8627-9e90ffd27c10 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:46.480705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 119/230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:01:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:46.487137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c65b5-afe3-4fc8-8627-9e90ffd27c10 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:46.537010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 112/231] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:01:46 2022] PUT /identity/v3/projects/f00dbca6b5e149df98838a84028da079/users/16c372374bf74737995c2046fa929fc7/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:46.565229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:46.612298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c65b5-afe3-4fc8-8627-9e90ffd27c10 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:46.653267 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c02cc8-b265-4376-8fcb-f85402093d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:46.654182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c02cc8-b265-4376-8fcb-f85402093d4c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:46.654495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c02cc8-b265-4376-8fcb-f85402093d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:46.654776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c02cc8-b265-4376-8fcb-f85402093d4c None admin] PATH_INFO: `/v3/users/16c372374bf74737995c2046fa929fc7` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:46.656751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c02cc8-b265-4376-8fcb-f85402093d4c None admin] RBAC: Authorizing `identity:get_user(user_id=16c372374bf74737995c2046fa929fc7)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:46.658595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c02cc8-b265-4376-8fcb-f85402093d4c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:46.660397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 120/232] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:46 2022] GET /identity/v3/users/16c372374bf74737995c2046fa929fc7 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:46.666301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304716be-ad50-472c-bbd5-aeb0dc09c31a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:46.759516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:46.814479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304716be-ad50-472c-bbd5-aeb0dc09c31a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:46.866922 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0dc975-07a1-416e-8300-2ad064dd1996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:46.867802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dc975-07a1-416e-8300-2ad064dd1996 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:46.868090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dc975-07a1-416e-8300-2ad064dd1996 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:46.868373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dc975-07a1-416e-8300-2ad064dd1996 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:46.869606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dc975-07a1-416e-8300-2ad064dd1996 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:46.871122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dc975-07a1-416e-8300-2ad064dd1996 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:46.873080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 113/233] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:46 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:46.879745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c02cc8-b265-4376-8fcb-f85402093d4c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:47.036890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c02cc8-b265-4376-8fcb-f85402093d4c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:47.078948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:47.088862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0233db6-464a-4edd-a71e-74b1b9b81289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:47.089779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233db6-464a-4edd-a71e-74b1b9b81289 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.090066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233db6-464a-4edd-a71e-74b1b9b81289 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.090339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0233db6-464a-4edd-a71e-74b1b9b81289 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079/users/16c372374bf74737995c2046fa929fc7/roles/13607cfcced34c8badf88b8c244d32c5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.092683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0233db6-464a-4edd-a71e-74b1b9b81289 None admin] RBAC: Authorizing `identity:create_grant(project_id=f00dbca6b5e149df98838a84028da079, user_id=16c372374bf74737995c2046fa929fc7, role_id=13607cfcced34c8badf88b8c244d32c5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:47.094247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0233db6-464a-4edd-a71e-74b1b9b81289 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:47.550699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 121/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:01:46 2022] PUT /identity/v3/projects/f00dbca6b5e149df98838a84028da079/users/16c372374bf74737995c2046fa929fc7/roles/13607cfcced34c8badf88b8c244d32c5 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:47.687899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b39b3-ab3f-4451-9eb9-f671a365c28e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.688407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b39b3-ab3f-4451-9eb9-f671a365c28e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.688906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b39b3-ab3f-4451-9eb9-f671a365c28e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.692052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 114/235] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:47.703084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffff309-ba70-46d0-952f-bbc012f07db1 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.703667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffff309-ba70-46d0-952f-bbc012f07db1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.704169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffff309-ba70-46d0-952f-bbc012f07db1 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.707610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ec8ce-8c82-43ac-b132-463bf6e9d106 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.708117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ec8ce-8c82-43ac-b132-463bf6e9d106 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.708603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ec8ce-8c82-43ac-b132-463bf6e9d106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.712064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 115/236] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:47.718683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5457e6-dda0-465e-8038-d19fcff31836 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.719187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5457e6-dda0-465e-8038-d19fcff31836 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.719739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5457e6-dda0-465e-8038-d19fcff31836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.763503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b77ec8ce-8c82-43ac-b132-463bf6e9d106 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:47.766076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b77ec8ce-8c82-43ac-b132-463bf6e9d106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:47.772545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4e5457e6-dda0-465e-8038-d19fcff31836 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:47.773795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4e5457e6-dda0-465e-8038-d19fcff31836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:47.885794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 122/237] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:47.887599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a269d-7532-4266-b205-05e4962737e5 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.887891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a269d-7532-4266-b205-05e4962737e5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.888173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a269d-7532-4266-b205-05e4962737e5 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.889511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 123/238] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:47.895690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d25fe-5fd0-4953-a4ac-cd07ae28f0c4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:47.895979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d25fe-5fd0-4953-a4ac-cd07ae28f0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:47.896260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d25fe-5fd0-4953-a4ac-cd07ae28f0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:47.900596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 116/239] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:47.914651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5457e6-dda0-465e-8038-d19fcff31836 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:47.919478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4c9d25fe-5fd0-4953-a4ac-cd07ae28f0c4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:47.920693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4c9d25fe-5fd0-4953-a4ac-cd07ae28f0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:47.990812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:48.005116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 124/240] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:48.006505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d25fe-5fd0-4953-a4ac-cd07ae28f0c4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.068689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5457e6-dda0-465e-8038-d19fcff31836 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.073522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:48.114392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11266782-2df8-4adf-9c01-b0b9c0bc85bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.115246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11266782-2df8-4adf-9c01-b0b9c0bc85bf None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.115583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11266782-2df8-4adf-9c01-b0b9c0bc85bf None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.115873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11266782-2df8-4adf-9c01-b0b9c0bc85bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.128652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11266782-2df8-4adf-9c01-b0b9c0bc85bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-11266782-2df8-4adf-9c01-b0b9c0bc85bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:48.137965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:48.139008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:48.140389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 117/241] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:48.141101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11266782-2df8-4adf-9c01-b0b9c0bc85bf None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.156954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d25fe-5fd0-4953-a4ac-cd07ae28f0c4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.211513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e599a-cd5c-48ba-a103-8afdcff4a5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.212907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e599a-cd5c-48ba-a103-8afdcff4a5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.213684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e599a-cd5c-48ba-a103-8afdcff4a5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.214547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e599a-cd5c-48ba-a103-8afdcff4a5d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.216519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e599a-cd5c-48ba-a103-8afdcff4a5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:48.218880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e599a-cd5c-48ba-a103-8afdcff4a5d8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:48.225321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 125/242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:48.226439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e599a-cd5c-48ba-a103-8afdcff4a5d8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.277356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11266782-2df8-4adf-9c01-b0b9c0bc85bf None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.301657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:48.322730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8acac5-451b-49d5-9b7c-0103384a9247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.323904 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8acac5-451b-49d5-9b7c-0103384a9247 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.324192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8acac5-451b-49d5-9b7c-0103384a9247 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.324486 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8acac5-451b-49d5-9b7c-0103384a9247 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.337906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8acac5-451b-49d5-9b7c-0103384a9247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5a8acac5-451b-49d5-9b7c-0103384a9247 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:48.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:48.350059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:48.350941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 118/243] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:48.351606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8acac5-451b-49d5-9b7c-0103384a9247 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.364375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e599a-cd5c-48ba-a103-8afdcff4a5d8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.387209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:48.408605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbc218-62de-4afa-8a64-71b4fd460da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.409281 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbc218-62de-4afa-8a64-71b4fd460da8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.409567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbc218-62de-4afa-8a64-71b4fd460da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.409842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbc218-62de-4afa-8a64-71b4fd460da8 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.410842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbc218-62de-4afa-8a64-71b4fd460da8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:48.411017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbc218-62de-4afa-8a64-71b4fd460da8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:48.412407 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbc218-62de-4afa-8a64-71b4fd460da8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:48.423435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 126/244] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:48.424293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbc218-62de-4afa-8a64-71b4fd460da8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.557585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8acac5-451b-49d5-9b7c-0103384a9247 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.558041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbc218-62de-4afa-8a64-71b4fd460da8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.602060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1689f-304b-4159-89d9-27476af366b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.602783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1689f-304b-4159-89d9-27476af366b4 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.603064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1689f-304b-4159-89d9-27476af366b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.603402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1689f-304b-4159-89d9-27476af366b4 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.604229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1689f-304b-4159-89d9-27476af366b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:48.605661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1689f-304b-4159-89d9-27476af366b4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:48.609953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1e752-1a8a-4fa5-b415-6f886e2e2ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.610867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1e752-1a8a-4fa5-b415-6f886e2e2ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.611317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1e752-1a8a-4fa5-b415-6f886e2e2ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.611743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1e752-1a8a-4fa5-b415-6f886e2e2ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.612646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1e752-1a8a-4fa5-b415-6f886e2e2ee4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:48.613454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1e752-1a8a-4fa5-b415-6f886e2e2ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:48.616071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1e752-1a8a-4fa5-b415-6f886e2e2ee4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:48.627245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 127/245] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:48.634161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1e752-1a8a-4fa5-b415-6f886e2e2ee4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.680963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:48.737767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 119/246] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:01:48 2022] POST /identity/v3/users => generated 293 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:48.739304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1689f-304b-4159-89d9-27476af366b4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.777855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1e752-1a8a-4fa5-b415-6f886e2e2ee4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:48.810193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:48.862192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09bf8-6ee6-455c-9eef-eadf1f221c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:48.863197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09bf8-6ee6-455c-9eef-eadf1f221c5f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:48.863479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09bf8-6ee6-455c-9eef-eadf1f221c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:48.863754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09bf8-6ee6-455c-9eef-eadf1f221c5f None admin] PATH_INFO: `/v3/users/d6b8241dd8334cf191bf335ba6e8c282` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:48.885156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c09bf8-6ee6-455c-9eef-eadf1f221c5f None admin] RBAC: Authorizing `identity:get_user(user_id=d6b8241dd8334cf191bf335ba6e8c282)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:48.886810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c09bf8-6ee6-455c-9eef-eadf1f221c5f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:48.888724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 128/247] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:48 2022] GET /identity/v3/users/d6b8241dd8334cf191bf335ba6e8c282 => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:48.898086 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09bf8-6ee6-455c-9eef-eadf1f221c5f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:48.995264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1689f-304b-4159-89d9-27476af366b4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:49.048145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a025d-b0c9-47e6-97fa-ab04516fba26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:49.051132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a025d-b0c9-47e6-97fa-ab04516fba26 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:49.052188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a025d-b0c9-47e6-97fa-ab04516fba26 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:49.052852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a025d-b0c9-47e6-97fa-ab04516fba26 None admin] PATH_INFO: `/v3/users/2b916df52426485fbd88bc259fc97953` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:49.056000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a025d-b0c9-47e6-97fa-ab04516fba26 None admin] RBAC: Authorizing `identity:get_user(user_id=2b916df52426485fbd88bc259fc97953)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:49.059202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a025d-b0c9-47e6-97fa-ab04516fba26 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:49.064853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 120/248] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:48 2022] GET /identity/v3/users/2b916df52426485fbd88bc259fc97953 => generated 297 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:49.070604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a025d-b0c9-47e6-97fa-ab04516fba26 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:49.083885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:49.089359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09bf8-6ee6-455c-9eef-eadf1f221c5f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:49.142423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad664b-755e-4e1b-957a-fa3309ce246c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:49.143285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad664b-755e-4e1b-957a-fa3309ce246c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:49.143680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad664b-755e-4e1b-957a-fa3309ce246c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:49.144019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad664b-755e-4e1b-957a-fa3309ce246c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:49.230689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:49.234778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a025d-b0c9-47e6-97fa-ab04516fba26 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:49.250719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad664b-755e-4e1b-957a-fa3309ce246c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:49.253177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad664b-755e-4e1b-957a-fa3309ce246c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:49.282014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988deef-992e-4b98-a05e-067da635bc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:49.286187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988deef-992e-4b98-a05e-067da635bc0a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:49.286607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988deef-992e-4b98-a05e-067da635bc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:49.286958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988deef-992e-4b98-a05e-067da635bc0a None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:49.288521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988deef-992e-4b98-a05e-067da635bc0a None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:49.290038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988deef-992e-4b98-a05e-067da635bc0a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:49.296654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 121/249] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:49 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:49.302499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988deef-992e-4b98-a05e-067da635bc0a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:49.383801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:49.389644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 129/250] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:49.396601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad664b-755e-4e1b-957a-fa3309ce246c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:49.445887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988deef-992e-4b98-a05e-067da635bc0a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:49.503521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93eb23b-2664-4ea9-b90f-3e4c86849cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:49.504829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eb23b-2664-4ea9-b90f-3e4c86849cab None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:49.505543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eb23b-2664-4ea9-b90f-3e4c86849cab None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:49.505703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eb23b-2664-4ea9-b90f-3e4c86849cab None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079/users/2b916df52426485fbd88bc259fc97953/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:49.509758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93eb23b-2664-4ea9-b90f-3e4c86849cab None admin] RBAC: Authorizing `identity:create_grant(project_id=f00dbca6b5e149df98838a84028da079, user_id=2b916df52426485fbd88bc259fc97953, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:49.512422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93eb23b-2664-4ea9-b90f-3e4c86849cab None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:49.568990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad664b-755e-4e1b-957a-fa3309ce246c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:49.608354 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 122/251] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:01:49 2022] PUT /identity/v3/projects/f00dbca6b5e149df98838a84028da079/users/2b916df52426485fbd88bc259fc97953/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:01:49.609893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea1ea1-835b-4dc9-a91d-c1a559cd5bab None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:49.610184 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea1ea1-835b-4dc9-a91d-c1a559cd5bab None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:49.610492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea1ea1-835b-4dc9-a91d-c1a559cd5bab None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:49.611897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 123/252] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:49.618913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806721be-2651-4da3-ab1a-dd39ecce02a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:49.619871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806721be-2651-4da3-ab1a-dd39ecce02a7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:49.620160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806721be-2651-4da3-ab1a-dd39ecce02a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:49.620435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806721be-2651-4da3-ab1a-dd39ecce02a7 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:49.623300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806721be-2651-4da3-ab1a-dd39ecce02a7 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:49.624996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806721be-2651-4da3-ab1a-dd39ecce02a7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:49.630018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 130/253] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:49 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:49.632941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40df99-6372-4ae9-9d34-64517cd686e0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:49.633233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40df99-6372-4ae9-9d34-64517cd686e0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:49.633522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40df99-6372-4ae9-9d34-64517cd686e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:49.643080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806721be-2651-4da3-ab1a-dd39ecce02a7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:49.675644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ca40df99-6372-4ae9-9d34-64517cd686e0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:49.676855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ca40df99-6372-4ae9-9d34-64517cd686e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:49.744676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:49.771746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:49.811335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806721be-2651-4da3-ab1a-dd39ecce02a7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:49.864958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 124/254] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:49.877846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3a91e-e7c8-4d50-9234-2e53d84b1b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:49.879024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3a91e-e7c8-4d50-9234-2e53d84b1b03 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:49.879537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3a91e-e7c8-4d50-9234-2e53d84b1b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:49.880033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3a91e-e7c8-4d50-9234-2e53d84b1b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:49.881113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3a91e-e7c8-4d50-9234-2e53d84b1b03 None admin] RBAC: Adding query filter params (role.id=6014fae253be445d9675cf76770c7b4c, scope.project.id=f00dbca6b5e149df98838a84028da079, user.id=d6b8241dd8334cf191bf335ba6e8c282) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:49.881677 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3a91e-e7c8-4d50-9234-2e53d84b1b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:49.883504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef3a91e-e7c8-4d50-9234-2e53d84b1b03 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:49.895193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 131/255] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:49 2022] GET /identity/v3/role_assignments?user.id=d6b8241dd8334cf191bf335ba6e8c282&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=6014fae253be445d9675cf76770c7b4c => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:49.902310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40df99-6372-4ae9-9d34-64517cd686e0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:50.072944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40df99-6372-4ae9-9d34-64517cd686e0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:50.125081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb91204-20b4-4d81-9683-d1fd20627a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:50.125973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb91204-20b4-4d81-9683-d1fd20627a42 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:50.126252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb91204-20b4-4d81-9683-d1fd20627a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:50.126555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb91204-20b4-4d81-9683-d1fd20627a42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:50.139039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb91204-20b4-4d81-9683-d1fd20627a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0cb91204-20b4-4d81-9683-d1fd20627a42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:50.147484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:50.148377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:50.149156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 125/256] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:01:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:50.156234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3a91e-e7c8-4d50-9234-2e53d84b1b03 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:50.278675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3a91e-e7c8-4d50-9234-2e53d84b1b03 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:50.291291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223f34c-b5ca-4e87-9066-e39f35a421b1 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:50.291618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223f34c-b5ca-4e87-9066-e39f35a421b1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:50.291851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223f34c-b5ca-4e87-9066-e39f35a421b1 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:50.293661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 126/257] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:50.299828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3185187-eee1-405f-9f7e-192aa0946da5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:50.300389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3185187-eee1-405f-9f7e-192aa0946da5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:50.300919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3185187-eee1-405f-9f7e-192aa0946da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:50.329954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b3185187-eee1-405f-9f7e-192aa0946da5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:50.332268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b3185187-eee1-405f-9f7e-192aa0946da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:50.346831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:50.352254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b99d3-c370-4a2e-8322-d67b590a66a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:50.352941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b99d3-c370-4a2e-8322-d67b590a66a5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:50.353230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b99d3-c370-4a2e-8322-d67b590a66a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:50.353514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b99d3-c370-4a2e-8322-d67b590a66a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:50.354272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b99d3-c370-4a2e-8322-d67b590a66a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:50.354612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b99d3-c370-4a2e-8322-d67b590a66a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:50.356019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b99d3-c370-4a2e-8322-d67b590a66a5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:50.367364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 132/258] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:50.373235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b99d3-c370-4a2e-8322-d67b590a66a5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:50.375338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:50.505085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 127/259] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:50.548050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3185187-eee1-405f-9f7e-192aa0946da5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:50.551162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b99d3-c370-4a2e-8322-d67b590a66a5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:50.619882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ee37-ccfc-459a-8daf-b88e82411046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:50.621047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ee37-ccfc-459a-8daf-b88e82411046 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:50.621493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ee37-ccfc-459a-8daf-b88e82411046 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:50.621906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847ee37-ccfc-459a-8daf-b88e82411046 None admin] PATH_INFO: `/v3/users/16c372374bf74737995c2046fa929fc7` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:50.625789 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847ee37-ccfc-459a-8daf-b88e82411046 None admin] RBAC: Authorizing `identity:get_user(user_id=16c372374bf74737995c2046fa929fc7)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:50.627567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847ee37-ccfc-459a-8daf-b88e82411046 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:50.629889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 133/260] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:50 2022] GET /identity/v3/users/16c372374bf74737995c2046fa929fc7 => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:50.635895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ee37-ccfc-459a-8daf-b88e82411046 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:50.760744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:50.816862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3185187-eee1-405f-9f7e-192aa0946da5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:50.827670 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847ee37-ccfc-459a-8daf-b88e82411046 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:50.887167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4f8b3-2d24-4cff-91d7-f2b2b513b601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:50.888377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4f8b3-2d24-4cff-91d7-f2b2b513b601 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:50.888823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4f8b3-2d24-4cff-91d7-f2b2b513b601 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:50.889106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4f8b3-2d24-4cff-91d7-f2b2b513b601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:50.919019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187bb5d-d17c-4e39-a7b5-6ff7294eb2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:50.919854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187bb5d-d17c-4e39-a7b5-6ff7294eb2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:50.920143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187bb5d-d17c-4e39-a7b5-6ff7294eb2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:50.920489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187bb5d-d17c-4e39-a7b5-6ff7294eb2a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:50.953255 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187bb5d-d17c-4e39-a7b5-6ff7294eb2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-d187bb5d-d17c-4e39-a7b5-6ff7294eb2a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:50.967487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:50.969876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:01:50.970849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:50.972953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 128/261] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:01:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:01:50.979604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187bb5d-d17c-4e39-a7b5-6ff7294eb2a9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:50.996612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:51.061022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4f8b3-2d24-4cff-91d7-f2b2b513b601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.063231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4f8b3-2d24-4cff-91d7-f2b2b513b601 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.218825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187bb5d-d17c-4e39-a7b5-6ff7294eb2a9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.249660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:51.315178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a7233-b538-449a-a4a2-e1cacfc6aeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.316423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a7233-b538-449a-a4a2-e1cacfc6aeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.316866 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a7233-b538-449a-a4a2-e1cacfc6aeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.317156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a7233-b538-449a-a4a2-e1cacfc6aeb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.318075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a7233-b538-449a-a4a2-e1cacfc6aeb2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:51.318393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a7233-b538-449a-a4a2-e1cacfc6aeb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.320337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a7233-b538-449a-a4a2-e1cacfc6aeb2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.334022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 129/262] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:01:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:51.340574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a7233-b538-449a-a4a2-e1cacfc6aeb2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.359286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 134/263] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:51.364653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4f8b3-2d24-4cff-91d7-f2b2b513b601 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.555839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4f8b3-2d24-4cff-91d7-f2b2b513b601 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.556604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:51.630639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a7233-b538-449a-a4a2-e1cacfc6aeb2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.633489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ceb67-85f9-4a42-a245-52b62bb46924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.634398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ceb67-85f9-4a42-a245-52b62bb46924 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.634778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ceb67-85f9-4a42-a245-52b62bb46924 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.635060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ceb67-85f9-4a42-a245-52b62bb46924 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.637861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ceb67-85f9-4a42-a245-52b62bb46924 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.639646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ceb67-85f9-4a42-a245-52b62bb46924 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.643572 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 135/264] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:51.649236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ceb67-85f9-4a42-a245-52b62bb46924 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.706806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:51.747709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3abc-7138-431d-ae10-c962dd0fa093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.748610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3abc-7138-431d-ae10-c962dd0fa093 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.749060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3abc-7138-431d-ae10-c962dd0fa093 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.749342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3abc-7138-431d-ae10-c962dd0fa093 None admin] PATH_INFO: `/v3/users/16c372374bf74737995c2046fa929fc7` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.755237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a3abc-7138-431d-ae10-c962dd0fa093 None admin] RBAC: Authorizing `identity:get_user(user_id=16c372374bf74737995c2046fa929fc7)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.756730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a3abc-7138-431d-ae10-c962dd0fa093 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.758500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 130/265] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/users/16c372374bf74737995c2046fa929fc7 => generated 283 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:51.764158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3abc-7138-431d-ae10-c962dd0fa093 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:51.812816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ceb67-85f9-4a42-a245-52b62bb46924 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.883160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7cac3-0a2c-448a-83c3-dd41523124c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.883848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7cac3-0a2c-448a-83c3-dd41523124c5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.884135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7cac3-0a2c-448a-83c3-dd41523124c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.884416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7cac3-0a2c-448a-83c3-dd41523124c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:51.885270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7cac3-0a2c-448a-83c3-dd41523124c5 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=f00dbca6b5e149df98838a84028da079, user.id=16c372374bf74737995c2046fa929fc7) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:51.885587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7cac3-0a2c-448a-83c3-dd41523124c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:51.887139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7cac3-0a2c-448a-83c3-dd41523124c5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:51.903987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 136/266] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/role_assignments?user.id=16c372374bf74737995c2046fa929fc7&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=bd295cb5d1194b469beff0381c6b23af => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:51.930659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3abc-7138-431d-ae10-c962dd0fa093 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:51.987396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32b44a-ee66-433c-b565-79de2fdf6ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:51.988071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32b44a-ee66-433c-b565-79de2fdf6ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:51.988361 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32b44a-ee66-433c-b565-79de2fdf6ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:51.988643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32b44a-ee66-433c-b565-79de2fdf6ce5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.004623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:52.082097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32b44a-ee66-433c-b565-79de2fdf6ce5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:52.083537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32b44a-ee66-433c-b565-79de2fdf6ce5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:52.216337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 131/267] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:52.222487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7cac3-0a2c-448a-83c3-dd41523124c5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:52.377451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:52.384287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7cac3-0a2c-448a-83c3-dd41523124c5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:52.427098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c24796-3aa7-40a2-99aa-8c95055a41cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:52.428019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24796-3aa7-40a2-99aa-8c95055a41cd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.428306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24796-3aa7-40a2-99aa-8c95055a41cd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.428582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c24796-3aa7-40a2-99aa-8c95055a41cd None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.429974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c24796-3aa7-40a2-99aa-8c95055a41cd None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:52.431878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c24796-3aa7-40a2-99aa-8c95055a41cd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:52.434042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 137/268] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:52 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:52.441524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32b44a-ee66-433c-b565-79de2fdf6ce5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:52.498188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:52.577874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32b44a-ee66-433c-b565-79de2fdf6ce5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:52.666029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bce4e-3b54-4a8c-9a5c-dd6a50091272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:52.666648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bce4e-3b54-4a8c-9a5c-dd6a50091272 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.666937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bce4e-3b54-4a8c-9a5c-dd6a50091272 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.667477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bce4e-3b54-4a8c-9a5c-dd6a50091272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.668327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bce4e-3b54-4a8c-9a5c-dd6a50091272 None admin] RBAC: Adding query filter params (role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=f00dbca6b5e149df98838a84028da079, user.id=16c372374bf74737995c2046fa929fc7) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:52.668645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bce4e-3b54-4a8c-9a5c-dd6a50091272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:52.670114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bce4e-3b54-4a8c-9a5c-dd6a50091272 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:52.677459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 132/269] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:52 2022] GET /identity/v3/role_assignments?user.id=16c372374bf74737995c2046fa929fc7&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:52.799073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f29fcc-4271-4533-834a-07cfacafdcc4 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.800327 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f29fcc-4271-4533-834a-07cfacafdcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.801092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f29fcc-4271-4533-834a-07cfacafdcc4 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.804284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 138/270] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:52.809887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab397bf2-44ca-43e5-a726-5378a4fcef6d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.810288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab397bf2-44ca-43e5-a726-5378a4fcef6d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.810561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab397bf2-44ca-43e5-a726-5378a4fcef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.824844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15a32-585e-4623-a2fc-7fdae56f0d9b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.825485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15a32-585e-4623-a2fc-7fdae56f0d9b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.826092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15a32-585e-4623-a2fc-7fdae56f0d9b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.828128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 139/271] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:52.835319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ab397bf2-44ca-43e5-a726-5378a4fcef6d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:52.836166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c05362-30c7-4954-93b2-6aa773926671 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:52.836656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c05362-30c7-4954-93b2-6aa773926671 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:52.836778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ab397bf2-44ca-43e5-a726-5378a4fcef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:52.837251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c05362-30c7-4954-93b2-6aa773926671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:52.865388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e3c05362-30c7-4954-93b2-6aa773926671 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:52.867329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e3c05362-30c7-4954-93b2-6aa773926671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:52.955490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 133/272] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:52.989434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 140/273] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:53.004681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab397bf2-44ca-43e5-a726-5378a4fcef6d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.012662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c05362-30c7-4954-93b2-6aa773926671 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.058001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:53.119545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:53.137080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c05362-30c7-4954-93b2-6aa773926671 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:53.157721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab397bf2-44ca-43e5-a726-5378a4fcef6d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:53.190475 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff64429-cef6-4d4c-8829-36a350ec4807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:53.192193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff64429-cef6-4d4c-8829-36a350ec4807 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.192861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff64429-cef6-4d4c-8829-36a350ec4807 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.193403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff64429-cef6-4d4c-8829-36a350ec4807 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.201678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3121f-4144-4f8f-891d-16e49eae2b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:53.204952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3121f-4144-4f8f-891d-16e49eae2b6c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.204952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3121f-4144-4f8f-891d-16e49eae2b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.205167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3121f-4144-4f8f-891d-16e49eae2b6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.208253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3121f-4144-4f8f-891d-16e49eae2b6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:53.210217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff64429-cef6-4d4c-8829-36a350ec4807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:53.210741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3121f-4144-4f8f-891d-16e49eae2b6c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:53.213234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 134/274] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:53.214936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2affd3-bf7d-41e4-ad6c-1905f6314852 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.215481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2affd3-bf7d-41e4-ad6c-1905f6314852 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.215962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2affd3-bf7d-41e4-ad6c-1905f6314852 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.217657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 135/275] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-fff64429-cef6-4d4c-8829-36a350ec4807 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:53.221964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:53.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:53.223754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 141/276] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:53.226122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2affd3-bf7d-41e4-ad6c-1905f6314852 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.226738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba4ed6-a6e4-4ef4-b540-02166529bea2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.227043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba4ed6-a6e4-4ef4-b540-02166529bea2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.227314 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba4ed6-a6e4-4ef4-b540-02166529bea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.245426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-52ba4ed6-a6e4-4ef4-b540-02166529bea2 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:53.246650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-52ba4ed6-a6e4-4ef4-b540-02166529bea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:53.334457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 142/277] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:53.334457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba4ed6-a6e4-4ef4-b540-02166529bea2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.347078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:53.351857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2affd3-bf7d-41e4-ad6c-1905f6314852 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:53.396415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeb540-af6f-4451-a822-f1a5fbbf50c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:53.397120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeb540-af6f-4451-a822-f1a5fbbf50c0 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.397405 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeb540-af6f-4451-a822-f1a5fbbf50c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.397685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeb540-af6f-4451-a822-f1a5fbbf50c0 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.398489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeb540-af6f-4451-a822-f1a5fbbf50c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:53.400006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeb540-af6f-4451-a822-f1a5fbbf50c0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:53.428696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:53.448429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba4ed6-a6e4-4ef4-b540-02166529bea2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:53.484498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50cf6a-33e1-459d-832a-54c277220fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:53.485065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50cf6a-33e1-459d-832a-54c277220fec None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.485349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50cf6a-33e1-459d-832a-54c277220fec None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.485628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50cf6a-33e1-459d-832a-54c277220fec None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.486392 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50cf6a-33e1-459d-832a-54c277220fec None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:53.486742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50cf6a-33e1-459d-832a-54c277220fec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:53.488176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f50cf6a-33e1-459d-832a-54c277220fec None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:53.497470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 143/278] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:53.498493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50cf6a-33e1-459d-832a-54c277220fec None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.536497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 136/279] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:01:53 2022] POST /identity/v3/users => generated 293 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:01:53.538006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a8f9a-ae72-44e8-a844-0d7ed9c56e45 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.538305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a8f9a-ae72-44e8-a844-0d7ed9c56e45 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.539231 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a8f9a-ae72-44e8-a844-0d7ed9c56e45 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.541105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 137/280] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:53.541934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a8f9a-ae72-44e8-a844-0d7ed9c56e45 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.663064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50cf6a-33e1-459d-832a-54c277220fec None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:53.669642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a8f9a-ae72-44e8-a844-0d7ed9c56e45 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:53.713461 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfbaa3-b7c7-467f-8cf8-39415ed7982f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:53.714360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfbaa3-b7c7-467f-8cf8-39415ed7982f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.714673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfbaa3-b7c7-467f-8cf8-39415ed7982f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.714950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfbaa3-b7c7-467f-8cf8-39415ed7982f None admin] PATH_INFO: `/v3/users/19dad1d877fa41089d9f214c69d1e27f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.718386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a748368-50cc-4931-ba11-54640db0fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:53.720628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a748368-50cc-4931-ba11-54640db0fc93 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.721111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a748368-50cc-4931-ba11-54640db0fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.721570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a748368-50cc-4931-ba11-54640db0fc93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.732030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfbaa3-b7c7-467f-8cf8-39415ed7982f None admin] RBAC: Authorizing `identity:get_user(user_id=19dad1d877fa41089d9f214c69d1e27f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:53.734205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfbaa3-b7c7-467f-8cf8-39415ed7982f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:53.735166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a748368-50cc-4931-ba11-54640db0fc93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:53.742274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 138/281] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:53 2022] GET /identity/v3/users/19dad1d877fa41089d9f214c69d1e27f => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:53.742274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8d4e-1418-4c69-9674-13813c89ea7a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.742274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8d4e-1418-4c69-9674-13813c89ea7a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.742274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8d4e-1418-4c69-9674-13813c89ea7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-3a748368-50cc-4931-ba11-54640db0fc93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:53.743326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:53.744202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:53.744987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:53.745574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 144/282] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:53.746339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a748368-50cc-4931-ba11-54640db0fc93 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.752299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:53.757618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c5ee8d4e-1418-4c69-9674-13813c89ea7a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:53.758867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c5ee8d4e-1418-4c69-9674-13813c89ea7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:53.770687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:53.846662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 139/283] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:53.847839 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8d4e-1418-4c69-9674-13813c89ea7a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.878004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a748368-50cc-4931-ba11-54640db0fc93 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:53.920241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24985ab-5743-4c3c-9a5a-0c259a05722b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:53.921035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24985ab-5743-4c3c-9a5a-0c259a05722b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:53.921312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24985ab-5743-4c3c-9a5a-0c259a05722b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:53.921733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24985ab-5743-4c3c-9a5a-0c259a05722b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:53.922619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24985ab-5743-4c3c-9a5a-0c259a05722b None admin] RBAC: Adding query filter params (role.id=60e46f07f6ab4fdd90ec5d32066e0faa, user.id=19dad1d877fa41089d9f214c69d1e27f, scope.system=all) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:53.922940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24985ab-5743-4c3c-9a5a-0c259a05722b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:53.924460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24985ab-5743-4c3c-9a5a-0c259a05722b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:53.930305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 145/284] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 12:01:53 2022] GET /identity/v3/role_assignments?user.id=19dad1d877fa41089d9f214c69d1e27f&scope.system=all&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:53.931220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24985ab-5743-4c3c-9a5a-0c259a05722b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:53.984656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8d4e-1418-4c69-9674-13813c89ea7a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:54.014186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:54.025648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f6f61-dd2c-4b5d-860e-333060053636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:54.026687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f6f61-dd2c-4b5d-860e-333060053636 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:54.027140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f6f61-dd2c-4b5d-860e-333060053636 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:54.028531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f6f61-dd2c-4b5d-860e-333060053636 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:54.028531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f6f61-dd2c-4b5d-860e-333060053636 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:54.028980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f6f61-dd2c-4b5d-860e-333060053636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:54.030814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f6f61-dd2c-4b5d-860e-333060053636 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:54.040710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 140/285] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:54.047307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f6f61-dd2c-4b5d-860e-333060053636 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:54.055553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24985ab-5743-4c3c-9a5a-0c259a05722b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:54.057194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:54.096343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d04760-a194-4648-b5fe-b49c2f34a5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:54.097308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d04760-a194-4648-b5fe-b49c2f34a5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:54.097639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d04760-a194-4648-b5fe-b49c2f34a5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:54.098310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d04760-a194-4648-b5fe-b49c2f34a5f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:54.112578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d04760-a194-4648-b5fe-b49c2f34a5f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-49d04760-a194-4648-b5fe-b49c2f34a5f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:54.122469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:54.123657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:54.124661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:54.126429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 146/286] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:54.133325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d04760-a194-4648-b5fe-b49c2f34a5f1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:54.266493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d04760-a194-4648-b5fe-b49c2f34a5f1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:54.267303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f6f61-dd2c-4b5d-860e-333060053636 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:54.298193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:54.307957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42788c-8214-4def-b3c3-6137ce6f6eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:54.313772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42788c-8214-4def-b3c3-6137ce6f6eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:54.313772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42788c-8214-4def-b3c3-6137ce6f6eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:54.313772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42788c-8214-4def-b3c3-6137ce6f6eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:54.313772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42788c-8214-4def-b3c3-6137ce6f6eb6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:54.313772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42788c-8214-4def-b3c3-6137ce6f6eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:54.313772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42788c-8214-4def-b3c3-6137ce6f6eb6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:54.329715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f2697-62db-45c6-ab1f-151e95f05ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:54.330521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 147/287] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:54.331555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f2697-62db-45c6-ab1f-151e95f05ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:54.332063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f2697-62db-45c6-ab1f-151e95f05ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:54.332445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42788c-8214-4def-b3c3-6137ce6f6eb6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:54.332769 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f2697-62db-45c6-ab1f-151e95f05ee9 None admin] PATH_INFO: `/v3/users/2b916df52426485fbd88bc259fc97953` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:54.334228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f2697-62db-45c6-ab1f-151e95f05ee9 None admin] RBAC: Authorizing `identity:get_user(user_id=2b916df52426485fbd88bc259fc97953)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:54.335876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f2697-62db-45c6-ab1f-151e95f05ee9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:54.337691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 141/288] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:54 2022] GET /identity/v3/users/2b916df52426485fbd88bc259fc97953 => generated 297 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:54.343294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f2697-62db-45c6-ab1f-151e95f05ee9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:54.482891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42788c-8214-4def-b3c3-6137ce6f6eb6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:54.524879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1daeab-e10c-4974-aed2-9a52a897ffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:54.526131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1daeab-e10c-4974-aed2-9a52a897ffd2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:54.526654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1daeab-e10c-4974-aed2-9a52a897ffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:54.527145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1daeab-e10c-4974-aed2-9a52a897ffd2 None admin] PATH_INFO: `/v3/users/d6b8241dd8334cf191bf335ba6e8c282` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:54.529194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1daeab-e10c-4974-aed2-9a52a897ffd2 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b8241dd8334cf191bf335ba6e8c282)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:54.531596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1daeab-e10c-4974-aed2-9a52a897ffd2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:54.533772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 148/289] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:54 2022] GET /identity/v3/users/d6b8241dd8334cf191bf335ba6e8c282 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:54.539059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1daeab-e10c-4974-aed2-9a52a897ffd2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:54.552582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:54.622612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:54.639939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f2697-62db-45c6-ab1f-151e95f05ee9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:54.676708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1daeab-e10c-4974-aed2-9a52a897ffd2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:54.709108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2f746-cab2-43ed-bc1f-d99b9adcc73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:54.710148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2f746-cab2-43ed-bc1f-d99b9adcc73f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:54.710656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2f746-cab2-43ed-bc1f-d99b9adcc73f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:54.711155 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2f746-cab2-43ed-bc1f-d99b9adcc73f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:54.752028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7760b-7c38-4574-99b5-2e9c81948877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:54.754377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7760b-7c38-4574-99b5-2e9c81948877 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:54.755101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7760b-7c38-4574-99b5-2e9c81948877 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:54.756367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7760b-7c38-4574-99b5-2e9c81948877 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:54.758827 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7760b-7c38-4574-99b5-2e9c81948877 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:54.762302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7760b-7c38-4574-99b5-2e9c81948877 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:54.765720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 149/290] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:54 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:54.771242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7760b-7c38-4574-99b5-2e9c81948877 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:54.812092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2f746-cab2-43ed-bc1f-d99b9adcc73f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:54.813888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2f746-cab2-43ed-bc1f-d99b9adcc73f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:54.941628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:54.946917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7760b-7c38-4574-99b5-2e9c81948877 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:54.978811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:54.989111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4623062-6f42-4026-bb44-4e020af42d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:54.990515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4623062-6f42-4026-bb44-4e020af42d65 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:54.991001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4623062-6f42-4026-bb44-4e020af42d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:54.991487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4623062-6f42-4026-bb44-4e020af42d65 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079/users/d6b8241dd8334cf191bf335ba6e8c282/roles/6014fae253be445d9675cf76770c7b4c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:54.993936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4623062-6f42-4026-bb44-4e020af42d65 None admin] RBAC: Authorizing `identity:create_grant(project_id=f00dbca6b5e149df98838a84028da079, user_id=d6b8241dd8334cf191bf335ba6e8c282, role_id=6014fae253be445d9675cf76770c7b4c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:54.995641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4623062-6f42-4026-bb44-4e020af42d65 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:55.011170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 142/291] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:55.019069 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2f746-cab2-43ed-bc1f-d99b9adcc73f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:55.131041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2f746-cab2-43ed-bc1f-d99b9adcc73f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:55.139378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 150/292] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:01:54 2022] PUT /identity/v3/projects/f00dbca6b5e149df98838a84028da079/users/d6b8241dd8334cf191bf335ba6e8c282/roles/6014fae253be445d9675cf76770c7b4c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:01:55.176194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70509284-c239-4df4-85e5-e3be7d00c3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:55.177390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509284-c239-4df4-85e5-e3be7d00c3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:55.177730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509284-c239-4df4-85e5-e3be7d00c3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:55.178018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70509284-c239-4df4-85e5-e3be7d00c3e7 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:55.179571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70509284-c239-4df4-85e5-e3be7d00c3e7 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:55.181192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70509284-c239-4df4-85e5-e3be7d00c3e7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:55.183548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 143/293] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:55 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:55.191633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4623062-6f42-4026-bb44-4e020af42d65 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:55.352756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4623062-6f42-4026-bb44-4e020af42d65 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:55.421581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d262-a7c6-4f9d-b0b0-664081fece5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:55.422294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d262-a7c6-4f9d-b0b0-664081fece5f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:55.422651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d262-a7c6-4f9d-b0b0-664081fece5f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:55.423084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d262-a7c6-4f9d-b0b0-664081fece5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:55.424579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d262-a7c6-4f9d-b0b0-664081fece5f None admin] RBAC: Adding query filter params (role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=f00dbca6b5e149df98838a84028da079, user.id=2b916df52426485fbd88bc259fc97953) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:55.425067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d262-a7c6-4f9d-b0b0-664081fece5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:55.427498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d262-a7c6-4f9d-b0b0-664081fece5f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:55.437138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 151/294] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:55 2022] GET /identity/v3/role_assignments?user.id=2b916df52426485fbd88bc259fc97953&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:56.279590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9cab3-24ab-41af-8dcb-54c41f696753 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.279590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9cab3-24ab-41af-8dcb-54c41f696753 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.279590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9cab3-24ab-41af-8dcb-54c41f696753 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.281417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 144/295] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:56.287462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8e373-af8c-4d79-80a0-ab84e94ef1ea None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.288371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8e373-af8c-4d79-80a0-ab84e94ef1ea None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.288874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8e373-af8c-4d79-80a0-ab84e94ef1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.308294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-60b8e373-af8c-4d79-80a0-ab84e94ef1ea None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:56.309851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-60b8e373-af8c-4d79-80a0-ab84e94ef1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:56.373472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:56.398610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 152/296] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:56.423544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9cab3-24ab-41af-8dcb-54c41f696753 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:56.428819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a5756-c67e-4f78-81fb-0fc795b3b5ae None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.429028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a5756-c67e-4f78-81fb-0fc795b3b5ae None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.429290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a5756-c67e-4f78-81fb-0fc795b3b5ae None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.430787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 153/297] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:01:56.439330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14adbf-0017-48cf-9078-55f7db7a0ff7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.439616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14adbf-0017-48cf-9078-55f7db7a0ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.443962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14adbf-0017-48cf-9078-55f7db7a0ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.458350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8a14adbf-0017-48cf-9078-55f7db7a0ff7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:56.459667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8a14adbf-0017-48cf-9078-55f7db7a0ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:56.482144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:56.563560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 154/298] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:56.568460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9cab3-24ab-41af-8dcb-54c41f696753 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:56.591013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14adbf-0017-48cf-9078-55f7db7a0ff7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:56.614637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb92e8e-3182-4257-be7e-dc8ed94f8dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:56.616465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb92e8e-3182-4257-be7e-dc8ed94f8dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.617089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb92e8e-3182-4257-be7e-dc8ed94f8dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.617678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb92e8e-3182-4257-be7e-dc8ed94f8dd2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.634396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb92e8e-3182-4257-be7e-dc8ed94f8dd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-bdb92e8e-3182-4257-be7e-dc8ed94f8dd2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:56.646518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:56.647602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:56.648395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:01:56.651175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 145/299] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:56.657487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb92e8e-3182-4257-be7e-dc8ed94f8dd2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:56.719315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14adbf-0017-48cf-9078-55f7db7a0ff7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:56.733634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:56.763829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5b560-206f-41b3-b3a6-59c3198ee380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:56.765201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5b560-206f-41b3-b3a6-59c3198ee380 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.765480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5b560-206f-41b3-b3a6-59c3198ee380 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.765987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5b560-206f-41b3-b3a6-59c3198ee380 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.767059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5b560-206f-41b3-b3a6-59c3198ee380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:56.768920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5b560-206f-41b3-b3a6-59c3198ee380 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:56.776361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 155/300] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:56.777790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6fdf6-fa44-42da-80e3-782236ed2d9b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.778079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6fdf6-fa44-42da-80e3-782236ed2d9b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.778360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6fdf6-fa44-42da-80e3-782236ed2d9b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.779952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 156/301] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:56.780683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6fdf6-fa44-42da-80e3-782236ed2d9b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:56.788049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb92e8e-3182-4257-be7e-dc8ed94f8dd2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:56.790737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:56.825010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783508f8-6a4d-4c6d-95f0-ef623b9a4462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:56.825591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783508f8-6a4d-4c6d-95f0-ef623b9a4462 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.825880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783508f8-6a4d-4c6d-95f0-ef623b9a4462 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.826165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783508f8-6a4d-4c6d-95f0-ef623b9a4462 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.828375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783508f8-6a4d-4c6d-95f0-ef623b9a4462 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:56.829116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783508f8-6a4d-4c6d-95f0-ef623b9a4462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:56.831033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783508f8-6a4d-4c6d-95f0-ef623b9a4462 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:56.840899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 146/302] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:56.842021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc838e1-1a74-414c-b56d-2c7a46dd577e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.842691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc838e1-1a74-414c-b56d-2c7a46dd577e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.842980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc838e1-1a74-414c-b56d-2c7a46dd577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.861257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4dc838e1-1a74-414c-b56d-2c7a46dd577e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:56.862520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4dc838e1-1a74-414c-b56d-2c7a46dd577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:56.905548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6fdf6-fa44-42da-80e3-782236ed2d9b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:56.946882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495bb51-1bdd-46da-9c13-2e9c31ac35a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:56.947871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495bb51-1bdd-46da-9c13-2e9c31ac35a9 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:56.947976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 147/303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:56.948882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc838e1-1a74-414c-b56d-2c7a46dd577e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:56.954917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495bb51-1bdd-46da-9c13-2e9c31ac35a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:56.956244 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495bb51-1bdd-46da-9c13-2e9c31ac35a9 None admin] PATH_INFO: `/v3/groups` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:56.957487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495bb51-1bdd-46da-9c13-2e9c31ac35a9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:56.960171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495bb51-1bdd-46da-9c13-2e9c31ac35a9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:57.050791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 157/304] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 12:01:56 2022] POST /identity/v3/groups => generated 229 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:01:57.052816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495bb51-1bdd-46da-9c13-2e9c31ac35a9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:57.066105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:57.073976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc838e1-1a74-414c-b56d-2c7a46dd577e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:57.106504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:57.117244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1d63-78dc-4fc6-9ae9-82602df86bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:57.118193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1d63-78dc-4fc6-9ae9-82602df86bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:57.118525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1d63-78dc-4fc6-9ae9-82602df86bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:57.125843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e1d63-78dc-4fc6-9ae9-82602df86bf3 None admin] PATH_INFO: `/v3/users/ceffc35d267c4cca95dac1e358752f39` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:57.137253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e1d63-78dc-4fc6-9ae9-82602df86bf3 None admin] RBAC: Authorizing `identity:get_user(user_id=ceffc35d267c4cca95dac1e358752f39)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:57.138883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24e1d63-78dc-4fc6-9ae9-82602df86bf3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:57.141724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 148/305] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:56 2022] GET /identity/v3/users/ceffc35d267c4cca95dac1e358752f39 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:57.147999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1d63-78dc-4fc6-9ae9-82602df86bf3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:57.193926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495bb51-1bdd-46da-9c13-2e9c31ac35a9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:57.244873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17793147-8baa-4539-9fa1-f8364e6465a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:57.246526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17793147-8baa-4539-9fa1-f8364e6465a4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:57.247006 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17793147-8baa-4539-9fa1-f8364e6465a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:57.247917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17793147-8baa-4539-9fa1-f8364e6465a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:57.264418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17793147-8baa-4539-9fa1-f8364e6465a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:57.273454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e1d63-78dc-4fc6-9ae9-82602df86bf3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-17793147-8baa-4539-9fa1-f8364e6465a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:57.274908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:57.275774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:01:57.276540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:57.284866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 158/306] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:57.285914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17793147-8baa-4539-9fa1-f8364e6465a4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:57.329592 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eedf3e-1219-4e3b-8243-9ead0bc6c432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:57.329592 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eedf3e-1219-4e3b-8243-9ead0bc6c432 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:57.329592 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eedf3e-1219-4e3b-8243-9ead0bc6c432 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:57.329592 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eedf3e-1219-4e3b-8243-9ead0bc6c432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:57.330665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eedf3e-1219-4e3b-8243-9ead0bc6c432 None admin] RBAC: Adding query filter params (role.id=6014fae253be445d9675cf76770c7b4c, user.id=ceffc35d267c4cca95dac1e358752f39, scope.system=all) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:57.331329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eedf3e-1219-4e3b-8243-9ead0bc6c432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:57.332375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eedf3e-1219-4e3b-8243-9ead0bc6c432 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:57.346102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 149/307] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 12:01:57 2022] GET /identity/v3/role_assignments?user.id=ceffc35d267c4cca95dac1e358752f39&scope.system=all&role.id=6014fae253be445d9675cf76770c7b4c => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:57.432011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:57.437628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17793147-8baa-4539-9fa1-f8364e6465a4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:57.480180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7d3c5-2f90-403e-a7a3-ded6d80ecb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:57.480862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7d3c5-2f90-403e-a7a3-ded6d80ecb4e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:57.481142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7d3c5-2f90-403e-a7a3-ded6d80ecb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:57.481415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7d3c5-2f90-403e-a7a3-ded6d80ecb4e None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:57.482194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7d3c5-2f90-403e-a7a3-ded6d80ecb4e None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:57.482678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7d3c5-2f90-403e-a7a3-ded6d80ecb4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:57.486454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7d3c5-2f90-403e-a7a3-ded6d80ecb4e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:57.496923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 159/308] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:57.503206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eedf3e-1219-4e3b-8243-9ead0bc6c432 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:57.524984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:57.633677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eedf3e-1219-4e3b-8243-9ead0bc6c432 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:57.671193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e7131-5703-4dd7-ac07-81a15f17d262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:57.672285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e7131-5703-4dd7-ac07-81a15f17d262 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:57.672572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e7131-5703-4dd7-ac07-81a15f17d262 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:57.672853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e7131-5703-4dd7-ac07-81a15f17d262 None admin] PATH_INFO: `/v3/users/19dad1d877fa41089d9f214c69d1e27f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:57.674109 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e7131-5703-4dd7-ac07-81a15f17d262 None admin] RBAC: Authorizing `identity:get_user(user_id=19dad1d877fa41089d9f214c69d1e27f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:57.675605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e7131-5703-4dd7-ac07-81a15f17d262 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:57.677282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 150/309] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:57 2022] GET /identity/v3/users/19dad1d877fa41089d9f214c69d1e27f => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:57.683242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7d3c5-2f90-403e-a7a3-ded6d80ecb4e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:57.824671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7d3c5-2f90-403e-a7a3-ded6d80ecb4e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:57.862713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cdb14-b23e-45d4-88a4-3dee18651301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:57.864003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cdb14-b23e-45d4-88a4-3dee18651301 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:57.864490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cdb14-b23e-45d4-88a4-3dee18651301 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:57.864967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4cdb14-b23e-45d4-88a4-3dee18651301 None admin] PATH_INFO: `/v3/system/users/19dad1d877fa41089d9f214c69d1e27f/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:57.866783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cdb14-b23e-45d4-88a4-3dee18651301 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=19dad1d877fa41089d9f214c69d1e27f, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:57.868426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4cdb14-b23e-45d4-88a4-3dee18651301 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:57.985613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3e5db-87a4-4fb6-b960-28c211345914 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:57.985944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3e5db-87a4-4fb6-b960-28c211345914 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:57.986122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3e5db-87a4-4fb6-b960-28c211345914 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:57.987541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 151/310] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:57.993670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34601a-8a07-41a6-80de-4ba47d2f0c90 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:57.993966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34601a-8a07-41a6-80de-4ba47d2f0c90 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:57.994252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34601a-8a07-41a6-80de-4ba47d2f0c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:58.004980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 160/311] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 12:01:57 2022] PUT /identity/v3/system/users/19dad1d877fa41089d9f214c69d1e27f/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:01:58.019396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cc34601a-8a07-41a6-80de-4ba47d2f0c90 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:58.020988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cc34601a-8a07-41a6-80de-4ba47d2f0c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:58.134918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 152/312] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:01:58.158867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cdb14-b23e-45d4-88a4-3dee18651301 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:58.192456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:58.283699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4cdb14-b23e-45d4-88a4-3dee18651301 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:58.321566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc06914-fd5a-48b4-8486-acb68f1d1349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:58.322775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc06914-fd5a-48b4-8486-acb68f1d1349 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:58.323297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc06914-fd5a-48b4-8486-acb68f1d1349 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:58.323852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc06914-fd5a-48b4-8486-acb68f1d1349 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:58.336410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc06914-fd5a-48b4-8486-acb68f1d1349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-efc06914-fd5a-48b4-8486-acb68f1d1349 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:01:58.345621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:01:58.346617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:01:58.347492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:01:58.350570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 161/313] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:01:58.359995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34601a-8a07-41a6-80de-4ba47d2f0c90 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:58.378938 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:58.486729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34601a-8a07-41a6-80de-4ba47d2f0c90 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:58.525390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0eb4b-62a8-48c4-9207-74b7e6d688f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:58.526504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0eb4b-62a8-48c4-9207-74b7e6d688f0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:58.526991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0eb4b-62a8-48c4-9207-74b7e6d688f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:58.527487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0eb4b-62a8-48c4-9207-74b7e6d688f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:58.528453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0eb4b-62a8-48c4-9207-74b7e6d688f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:58.528962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0eb4b-62a8-48c4-9207-74b7e6d688f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:58.530908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0eb4b-62a8-48c4-9207-74b7e6d688f0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:58.541023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 153/314] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:01:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:58.549906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc06914-fd5a-48b4-8486-acb68f1d1349 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:58.675618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:58.697206 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc06914-fd5a-48b4-8486-acb68f1d1349 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:58.735014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e2f2-b683-475d-937c-7a44f37ff6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:58.735914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e2f2-b683-475d-937c-7a44f37ff6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:58.736198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e2f2-b683-475d-937c-7a44f37ff6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:58.736476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e2f2-b683-475d-937c-7a44f37ff6b8 None admin] PATH_INFO: `/v3/users/d6b8241dd8334cf191bf335ba6e8c282` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:58.737722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8e2f2-b683-475d-937c-7a44f37ff6b8 None admin] RBAC: Authorizing `identity:get_user(user_id=d6b8241dd8334cf191bf335ba6e8c282)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:58.739225 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8e2f2-b683-475d-937c-7a44f37ff6b8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:58.741034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 162/315] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:01:58 2022] GET /identity/v3/users/d6b8241dd8334cf191bf335ba6e8c282 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:58.747494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0eb4b-62a8-48c4-9207-74b7e6d688f0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:58.846030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:58.869476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0eb4b-62a8-48c4-9207-74b7e6d688f0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:58.905748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b534e-26ad-4432-a68d-ae3a5105dfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:58.906424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b534e-26ad-4432-a68d-ae3a5105dfbe None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:58.906733 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b534e-26ad-4432-a68d-ae3a5105dfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:58.907426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b534e-26ad-4432-a68d-ae3a5105dfbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:58.981101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b534e-26ad-4432-a68d-ae3a5105dfbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:58.982741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b534e-26ad-4432-a68d-ae3a5105dfbe None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:59.071347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:59.091307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 154/316] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:01:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:01:59.097413 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e2f2-b683-475d-937c-7a44f37ff6b8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:59.209904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e2f2-b683-475d-937c-7a44f37ff6b8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:59.249610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bba5e3-85e5-42b7-b309-cf46b7a77e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:59.250510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba5e3-85e5-42b7-b309-cf46b7a77e9b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:59.250795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba5e3-85e5-42b7-b309-cf46b7a77e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:59.251068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bba5e3-85e5-42b7-b309-cf46b7a77e9b None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:59.252279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba5e3-85e5-42b7-b309-cf46b7a77e9b None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:59.253737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bba5e3-85e5-42b7-b309-cf46b7a77e9b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:59.255654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 163/317] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:01:59 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:01:59.261333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b534e-26ad-4432-a68d-ae3a5105dfbe None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:59.375630 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b534e-26ad-4432-a68d-ae3a5105dfbe None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:59.412645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a430d5b9-5f6a-48d2-83f0-c471838c979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:59.413309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430d5b9-5f6a-48d2-83f0-c471838c979d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:59.413594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430d5b9-5f6a-48d2-83f0-c471838c979d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:59.413875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a430d5b9-5f6a-48d2-83f0-c471838c979d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:59.414749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430d5b9-5f6a-48d2-83f0-c471838c979d None admin] RBAC: Adding query filter params (role.id=6014fae253be445d9675cf76770c7b4c, scope.project.id=f00dbca6b5e149df98838a84028da079, user.id=d6b8241dd8334cf191bf335ba6e8c282) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:01:59.415072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430d5b9-5f6a-48d2-83f0-c471838c979d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:59.416526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a430d5b9-5f6a-48d2-83f0-c471838c979d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:01:59.424532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 155/318] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:01:59 2022] GET /identity/v3/role_assignments?user.id=d6b8241dd8334cf191bf335ba6e8c282&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=6014fae253be445d9675cf76770c7b4c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:01:59.709723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019edd6-4b00-4195-891b-60b9ca476e32 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:59.710618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019edd6-4b00-4195-891b-60b9ca476e32 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:59.711139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019edd6-4b00-4195-891b-60b9ca476e32 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:59.712975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 164/319] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:59.719802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea815ba-418f-4ccf-aee8-bef9afb62e9d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:59.720101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea815ba-418f-4ccf-aee8-bef9afb62e9d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:59.720492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea815ba-418f-4ccf-aee8-bef9afb62e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:59.738930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4ea815ba-418f-4ccf-aee8-bef9afb62e9d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:59.740145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4ea815ba-418f-4ccf-aee8-bef9afb62e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:59.778847 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:59.814573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 156/320] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:59.819734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b553e-7636-4f9e-a86a-a1ffd411002b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:59.820097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b553e-7636-4f9e-a86a-a1ffd411002b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:59.820451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b553e-7636-4f9e-a86a-a1ffd411002b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:59.821890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 165/321] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:01:59.828408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23cd8-0c87-4576-9117-c75a88e908cc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:59.828705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23cd8-0c87-4576-9117-c75a88e908cc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:59.828987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23cd8-0c87-4576-9117-c75a88e908cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:59.837173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b553e-7636-4f9e-a86a-a1ffd411002b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:59.846204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-bce23cd8-0c87-4576-9117-c75a88e908cc None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:01:59.847434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-bce23cd8-0c87-4576-9117-c75a88e908cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:01:59.861629 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:01:59.918570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 157/322] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:01:59.941454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce23cd8-0c87-4576-9117-c75a88e908cc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:01:59.951005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b553e-7636-4f9e-a86a-a1ffd411002b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:01:59.989572 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6faca-4c0f-4b0e-b4f5-5cac913e6a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:01:59.990290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6faca-4c0f-4b0e-b4f5-5cac913e6a4e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:01:59.990605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6faca-4c0f-4b0e-b4f5-5cac913e6a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:01:59.990881 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6faca-4c0f-4b0e-b4f5-5cac913e6a4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:01:59.992161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6faca-4c0f-4b0e-b4f5-5cac913e6a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:01:59.993739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6faca-4c0f-4b0e-b4f5-5cac913e6a4e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:00.001381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 166/323] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:01:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:00.002287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6faca-4c0f-4b0e-b4f5-5cac913e6a4e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:00.053344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:00.057786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce23cd8-0c87-4576-9117-c75a88e908cc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:00.108107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edde41f-f75f-47c6-9968-1f853591b2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:00.109009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde41f-f75f-47c6-9968-1f853591b2fb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.109366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde41f-f75f-47c6-9968-1f853591b2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.109958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edde41f-f75f-47c6-9968-1f853591b2fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.110649 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:00.121523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edde41f-f75f-47c6-9968-1f853591b2fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:00.122924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6faca-4c0f-4b0e-b4f5-5cac913e6a4e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0edde41f-f75f-47c6-9968-1f853591b2fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:00.130565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:00.131541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:00.132353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 158/324] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:01:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:00.137663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edde41f-f75f-47c6-9968-1f853591b2fb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:00.164126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1c11-0748-4c4f-975f-b9ef5df53e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:00.164795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1c11-0748-4c4f-975f-b9ef5df53e0b None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.165078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1c11-0748-4c4f-975f-b9ef5df53e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.165350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1c11-0748-4c4f-975f-b9ef5df53e0b None admin] PATH_INFO: `/v3/groups` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.166039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1c11-0748-4c4f-975f-b9ef5df53e0b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:00.167832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1c11-0748-4c4f-975f-b9ef5df53e0b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:00.253163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edde41f-f75f-47c6-9968-1f853591b2fb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:00.288131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6d9da-4274-44c1-85e4-0376e39ebdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:00.288792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6d9da-4274-44c1-85e4-0376e39ebdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.289081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6d9da-4274-44c1-85e4-0376e39ebdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.289351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6d9da-4274-44c1-85e4-0376e39ebdc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.290097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6d9da-4274-44c1-85e4-0376e39ebdc3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:00.290401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6d9da-4274-44c1-85e4-0376e39ebdc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:00.291787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6d9da-4274-44c1-85e4-0376e39ebdc3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:00.302283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 159/325] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:02:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:00.303156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 167/326] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 19 12:02:00 2022] POST /identity/v3/groups => generated 226 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:02:00.313048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6d9da-4274-44c1-85e4-0376e39ebdc3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:00.344075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:00.383944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76061c-a56b-43b0-96a6-1fb61015139c None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.384350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76061c-a56b-43b0-96a6-1fb61015139c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.384636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76061c-a56b-43b0-96a6-1fb61015139c None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.386243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 168/327] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:00.392530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902dfa9-248b-4aff-9f75-5b69241a6c3e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.392840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902dfa9-248b-4aff-9f75-5b69241a6c3e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.393104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902dfa9-248b-4aff-9f75-5b69241a6c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.411961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8902dfa9-248b-4aff-9f75-5b69241a6c3e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:00.413262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8902dfa9-248b-4aff-9f75-5b69241a6c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:00.438389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6d9da-4274-44c1-85e4-0376e39ebdc3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:00.479686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7a51d-8bc2-4efc-a254-a929d43ae3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:00.480574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7a51d-8bc2-4efc-a254-a929d43ae3cc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.480859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7a51d-8bc2-4efc-a254-a929d43ae3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.481132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7a51d-8bc2-4efc-a254-a929d43ae3cc None admin] PATH_INFO: `/v3/users/ceffc35d267c4cca95dac1e358752f39` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.482392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7a51d-8bc2-4efc-a254-a929d43ae3cc None admin] RBAC: Authorizing `identity:get_user(user_id=ceffc35d267c4cca95dac1e358752f39)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:00.484882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7a51d-8bc2-4efc-a254-a929d43ae3cc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:00.486834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 160/328] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:02:00 2022] GET /identity/v3/users/ceffc35d267c4cca95dac1e358752f39 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:00.492507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7a51d-8bc2-4efc-a254-a929d43ae3cc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:00.495242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 169/329] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:00.519658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902dfa9-248b-4aff-9f75-5b69241a6c3e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:00.601130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:00.615540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7a51d-8bc2-4efc-a254-a929d43ae3cc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:00.704634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:00.736001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb8575-bca9-4855-a332-68e0f9c893c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:00.736979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb8575-bca9-4855-a332-68e0f9c893c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.737267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb8575-bca9-4855-a332-68e0f9c893c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.737546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb8575-bca9-4855-a332-68e0f9c893c5 None admin] PATH_INFO: `/v3/system/users/ceffc35d267c4cca95dac1e358752f39/roles/6014fae253be445d9675cf76770c7b4c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.739142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb8575-bca9-4855-a332-68e0f9c893c5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ceffc35d267c4cca95dac1e358752f39, role_id=6014fae253be445d9675cf76770c7b4c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:00.740608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb8575-bca9-4855-a332-68e0f9c893c5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:00.740858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902dfa9-248b-4aff-9f75-5b69241a6c3e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:00.782841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81c12e-c2d7-4860-bc06-67eb91d3445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:00.783579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81c12e-c2d7-4860-bc06-67eb91d3445f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.783865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81c12e-c2d7-4860-bc06-67eb91d3445f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.784139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81c12e-c2d7-4860-bc06-67eb91d3445f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.797880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81c12e-c2d7-4860-bc06-67eb91d3445f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-aa81c12e-c2d7-4860-bc06-67eb91d3445f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:00.805801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:00.806817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:00.807742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 170/330] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:00.813431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81c12e-c2d7-4860-bc06-67eb91d3445f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:00.832026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 161/331] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 19 12:02:00 2022] PUT /identity/v3/system/users/ceffc35d267c4cca95dac1e358752f39/roles/6014fae253be445d9675cf76770c7b4c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:02:00.940602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81c12e-c2d7-4860-bc06-67eb91d3445f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:00.986395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dbe22-163a-4bcb-875b-171336551ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:00.987148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dbe22-163a-4bcb-875b-171336551ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:00.987437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dbe22-163a-4bcb-875b-171336551ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:00.987719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dbe22-163a-4bcb-875b-171336551ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:00.988501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dbe22-163a-4bcb-875b-171336551ce8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:00.988810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dbe22-163a-4bcb-875b-171336551ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:00.990332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dbe22-163a-4bcb-875b-171336551ce8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:00.996504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:01.003641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 171/332] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:02:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:01.009727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb8575-bca9-4855-a332-68e0f9c893c5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:01.142901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb8575-bca9-4855-a332-68e0f9c893c5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:01.180467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad380675-8c6a-4d6c-9472-36ab8a036109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:01.181747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad380675-8c6a-4d6c-9472-36ab8a036109 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:01.182238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad380675-8c6a-4d6c-9472-36ab8a036109 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:01.182743 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad380675-8c6a-4d6c-9472-36ab8a036109 None admin] PATH_INFO: `/v3/users/19dad1d877fa41089d9f214c69d1e27f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:01.184197 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad380675-8c6a-4d6c-9472-36ab8a036109 None admin] RBAC: Authorizing `identity:get_user(user_id=19dad1d877fa41089d9f214c69d1e27f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:01.186433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad380675-8c6a-4d6c-9472-36ab8a036109 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:01.188768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 162/333] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:02:01 2022] GET /identity/v3/users/19dad1d877fa41089d9f214c69d1e27f => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:01.199573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dbe22-163a-4bcb-875b-171336551ce8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:01.338447 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dbe22-163a-4bcb-875b-171336551ce8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:01.387890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a60f35-88ca-446a-bedf-f3ea336b46df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:01.388574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a60f35-88ca-446a-bedf-f3ea336b46df None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:01.388878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a60f35-88ca-446a-bedf-f3ea336b46df None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:01.389159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a60f35-88ca-446a-bedf-f3ea336b46df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:01.390005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a60f35-88ca-446a-bedf-f3ea336b46df None admin] RBAC: Adding query filter params (role.id=60e46f07f6ab4fdd90ec5d32066e0faa, user.id=19dad1d877fa41089d9f214c69d1e27f, scope.system=all) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:01.390320 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a60f35-88ca-446a-bedf-f3ea336b46df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:01.391853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a60f35-88ca-446a-bedf-f3ea336b46df None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:01.406115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 172/334] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 12:02:01 2022] GET /identity/v3/role_assignments?user.id=19dad1d877fa41089d9f214c69d1e27f&scope.system=all&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:03.490311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd03a2d-93db-46a0-8b80-fcdcffffd4f0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:03.491388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd03a2d-93db-46a0-8b80-fcdcffffd4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:03.491881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd03a2d-93db-46a0-8b80-fcdcffffd4f0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:03.493800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 163/335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:03.499430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790fe4f-db71-4675-80b9-7c4822824629 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:03.500435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790fe4f-db71-4675-80b9-7c4822824629 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:03.500926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790fe4f-db71-4675-80b9-7c4822824629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:03.526116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7790fe4f-db71-4675-80b9-7c4822824629 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:03.527757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7790fe4f-db71-4675-80b9-7c4822824629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:03.695194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:03.721051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 173/336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:03.758139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd03a2d-93db-46a0-8b80-fcdcffffd4f0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:03.791834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:03.901083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42107837-b7f2-4ecc-8b8d-ef0b0360e50f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:03.901788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42107837-b7f2-4ecc-8b8d-ef0b0360e50f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:03.902280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42107837-b7f2-4ecc-8b8d-ef0b0360e50f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:03.906087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 174/337] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:03.911983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36ada7-20be-46d2-a32f-e43f32bbe4cb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:03.912573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36ada7-20be-46d2-a32f-e43f32bbe4cb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:03.913077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36ada7-20be-46d2-a32f-e43f32bbe4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:03.921831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd03a2d-93db-46a0-8b80-fcdcffffd4f0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:03.946491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2a36ada7-20be-46d2-a32f-e43f32bbe4cb None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:03.947149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2a36ada7-20be-46d2-a32f-e43f32bbe4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:03.985619 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed4602-82ff-4038-af28-f2c0c93c166d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:03.986450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed4602-82ff-4038-af28-f2c0c93c166d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:03.986744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed4602-82ff-4038-af28-f2c0c93c166d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:03.987032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed4602-82ff-4038-af28-f2c0c93c166d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.009105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed4602-82ff-4038-af28-f2c0c93c166d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-82ed4602-82ff-4038-af28-f2c0c93c166d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:04.037337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:04.038275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:04.039084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:04.040215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 164/338] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:04.046567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a7068-911a-4fb8-a609-95eec763d722 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.047218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a7068-911a-4fb8-a609-95eec763d722 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.047734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a7068-911a-4fb8-a609-95eec763d722 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.049431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 165/339] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:04.050448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a7068-911a-4fb8-a609-95eec763d722 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.088898 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 175/340] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:04.090265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0d276-2fe8-4e15-94c3-525be297165e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.090566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0d276-2fe8-4e15-94c3-525be297165e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.090851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0d276-2fe8-4e15-94c3-525be297165e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.121626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7ed0d276-2fe8-4e15-94c3-525be297165e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:04.123113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7ed0d276-2fe8-4e15-94c3-525be297165e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:04.218798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:04.264428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a7068-911a-4fb8-a609-95eec763d722 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:04.283820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 176/341] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:04.284610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0d276-2fe8-4e15-94c3-525be297165e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.334036 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9adb2-eeb1-46bb-893e-86a5db73e70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:04.336188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9adb2-eeb1-46bb-893e-86a5db73e70f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.336478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9adb2-eeb1-46bb-893e-86a5db73e70f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.336756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9adb2-eeb1-46bb-893e-86a5db73e70f None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.337539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9adb2-eeb1-46bb-893e-86a5db73e70f None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:04.337854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9adb2-eeb1-46bb-893e-86a5db73e70f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:04.339289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9adb2-eeb1-46bb-893e-86a5db73e70f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:04.367721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 166/342] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:02:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:04.368679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9adb2-eeb1-46bb-893e-86a5db73e70f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.398155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:04.491150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9adb2-eeb1-46bb-893e-86a5db73e70f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:04.527636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0d276-2fe8-4e15-94c3-525be297165e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:04.538448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4c3a-c274-4d3e-895c-645dca865e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:04.538448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4c3a-c274-4d3e-895c-645dca865e2f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.538744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4c3a-c274-4d3e-895c-645dca865e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.538813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e4c3a-c274-4d3e-895c-645dca865e2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.560200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e4c3a-c274-4d3e-895c-645dca865e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4b8e4c3a-c274-4d3e-895c-645dca865e2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:04.572955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:04.573885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:04.574712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 167/343] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:02:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:04.575844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4c3a-c274-4d3e-895c-645dca865e2f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.576367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434b264-3162-4a6e-b086-dce31d5d0510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:04.577430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434b264-3162-4a6e-b086-dce31d5d0510 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.577934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434b264-3162-4a6e-b086-dce31d5d0510 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.578561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434b264-3162-4a6e-b086-dce31d5d0510 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.581064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:04.592721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434b264-3162-4a6e-b086-dce31d5d0510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9434b264-3162-4a6e-b086-dce31d5d0510 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:04.601124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:04.602247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 12:02:04.603295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:04.605834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 177/344] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:04.607945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79542c-d666-4bd4-aeb0-d1286032b7bc None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.608614 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79542c-d666-4bd4-aeb0-d1286032b7bc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.609066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79542c-d666-4bd4-aeb0-d1286032b7bc None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.610682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 178/345] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:04.615638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79542c-d666-4bd4-aeb0-d1286032b7bc None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.699156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e4c3a-c274-4d3e-895c-645dca865e2f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:04.734516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79542c-d666-4bd4-aeb0-d1286032b7bc None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:04.739046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836c523-6891-4d8d-a610-6241919d915f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:04.739835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836c523-6891-4d8d-a610-6241919d915f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.740260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836c523-6891-4d8d-a610-6241919d915f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.740635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836c523-6891-4d8d-a610-6241919d915f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.749369 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:04.781639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae07636-6f56-4037-a8cf-a97e7513ac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:04.782235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07636-6f56-4037-a8cf-a97e7513ac1e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.782559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07636-6f56-4037-a8cf-a97e7513ac1e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.782841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae07636-6f56-4037-a8cf-a97e7513ac1e None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.783603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae07636-6f56-4037-a8cf-a97e7513ac1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:04.783919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae07636-6f56-4037-a8cf-a97e7513ac1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:04.785323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae07636-6f56-4037-a8cf-a97e7513ac1e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:04.796386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 179/346] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:04.797546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae07636-6f56-4037-a8cf-a97e7513ac1e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.832397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836c523-6891-4d8d-a610-6241919d915f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:04.834572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836c523-6891-4d8d-a610-6241919d915f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:04.837829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:04.914371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae07636-6f56-4037-a8cf-a97e7513ac1e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:04.957166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e56d3-074f-481d-826d-ee3253987700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:04.957635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 168/347] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:04.957999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e56d3-074f-481d-826d-ee3253987700 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.958356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e56d3-074f-481d-826d-ee3253987700 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.958735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e56d3-074f-481d-826d-ee3253987700 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.959558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e56d3-074f-481d-826d-ee3253987700 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:04.959938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e56d3-074f-481d-826d-ee3253987700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:04.960175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b241c7-0ffb-4925-a9ba-62d793c1ffc4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:04.960644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b241c7-0ffb-4925-a9ba-62d793c1ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:04.961086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b241c7-0ffb-4925-a9ba-62d793c1ffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:04.966068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e56d3-074f-481d-826d-ee3253987700 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:04.976999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 180/348] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:02:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:04.977969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e56d3-074f-481d-826d-ee3253987700 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:04.984611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-94b241c7-0ffb-4925-a9ba-62d793c1ffc4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:04.986981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-94b241c7-0ffb-4925-a9ba-62d793c1ffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:05.014373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:05.088967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 169/349] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:05.090142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b241c7-0ffb-4925-a9ba-62d793c1ffc4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:05.120777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e56d3-074f-481d-826d-ee3253987700 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:05.152813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:05.169960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7efccd-c5ff-46eb-a4e0-ff821932bac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:05.170701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7efccd-c5ff-46eb-a4e0-ff821932bac9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.171153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7efccd-c5ff-46eb-a4e0-ff821932bac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.171454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7efccd-c5ff-46eb-a4e0-ff821932bac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.263142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7efccd-c5ff-46eb-a4e0-ff821932bac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:05.265642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7efccd-c5ff-46eb-a4e0-ff821932bac9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:05.272643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b241c7-0ffb-4925-a9ba-62d793c1ffc4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:05.312972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:05.317043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b6704-e1de-489b-a460-d2f9a0dd7d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:05.318199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b6704-e1de-489b-a460-d2f9a0dd7d82 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.318607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b6704-e1de-489b-a460-d2f9a0dd7d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.319087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b6704-e1de-489b-a460-d2f9a0dd7d82 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.320518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b6704-e1de-489b-a460-d2f9a0dd7d82 None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:05.324195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b6704-e1de-489b-a460-d2f9a0dd7d82 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:05.328003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 170/350] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:05 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:05.329084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b6704-e1de-489b-a460-d2f9a0dd7d82 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:05.414366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 181/351] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:05.421282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7efccd-c5ff-46eb-a4e0-ff821932bac9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:05.463860 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b6704-e1de-489b-a460-d2f9a0dd7d82 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:05.465482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:05.504039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37732553-1700-470f-bf2c-e49199ab1f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:05.504920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37732553-1700-470f-bf2c-e49199ab1f11 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.505204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37732553-1700-470f-bf2c-e49199ab1f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.505480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37732553-1700-470f-bf2c-e49199ab1f11 None admin] PATH_INFO: `/v3/users/ceffc35d267c4cca95dac1e358752f39` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.506897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37732553-1700-470f-bf2c-e49199ab1f11 None admin] RBAC: Authorizing `identity:get_user(user_id=ceffc35d267c4cca95dac1e358752f39)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:05.508374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37732553-1700-470f-bf2c-e49199ab1f11 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:05.510253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 171/352] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:02:05 2022] GET /identity/v3/users/ceffc35d267c4cca95dac1e358752f39 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:05.511439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908a5e6-4a4d-455f-9c3d-cc4b24885de5 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.511743 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908a5e6-4a4d-455f-9c3d-cc4b24885de5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.512325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908a5e6-4a4d-455f-9c3d-cc4b24885de5 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.515204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 172/353] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:05.515961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908a5e6-4a4d-455f-9c3d-cc4b24885de5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:05.556749 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7efccd-c5ff-46eb-a4e0-ff821932bac9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:05.595718 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:05.600962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f71e-11b7-44a8-b6e4-ffbf3007042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:05.602273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f71e-11b7-44a8-b6e4-ffbf3007042c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.602273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f71e-11b7-44a8-b6e4-ffbf3007042c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.602487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f71e-11b7-44a8-b6e4-ffbf3007042c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.615117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2f71e-11b7-44a8-b6e4-ffbf3007042c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e7c2f71e-11b7-44a8-b6e4-ffbf3007042c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:05.623445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:05.624378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:05.625170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 182/354] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:05.625814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f71e-11b7-44a8-b6e4-ffbf3007042c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:05.658666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908a5e6-4a4d-455f-9c3d-cc4b24885de5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:05.699120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d4a3a-65ce-421d-a48c-0e9da29498c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:05.700039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d4a3a-65ce-421d-a48c-0e9da29498c5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.700369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d4a3a-65ce-421d-a48c-0e9da29498c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.700651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d4a3a-65ce-421d-a48c-0e9da29498c5 None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.713789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d4a3a-65ce-421d-a48c-0e9da29498c5 None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:05.716590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d4a3a-65ce-421d-a48c-0e9da29498c5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:05.720449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 173/355] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:05 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:05.722880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0447ad-f9ae-49e5-a5d2-cad3ed47e3c5 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.723412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0447ad-f9ae-49e5-a5d2-cad3ed47e3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.723917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0447ad-f9ae-49e5-a5d2-cad3ed47e3c5 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.727432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 174/356] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:05.727904 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0447ad-f9ae-49e5-a5d2-cad3ed47e3c5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:05.756929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f71e-11b7-44a8-b6e4-ffbf3007042c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:05.774182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:05.799495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7370592-3407-4ab4-975c-5cc380ef4e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:05.800360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7370592-3407-4ab4-975c-5cc380ef4e2d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.800643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7370592-3407-4ab4-975c-5cc380ef4e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.800918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7370592-3407-4ab4-975c-5cc380ef4e2d None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.802128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7370592-3407-4ab4-975c-5cc380ef4e2d None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:05.803647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7370592-3407-4ab4-975c-5cc380ef4e2d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:05.805343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 183/357] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:05 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:05.806621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157cae3-9501-4a08-bfc1-1de676a091d4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.806922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157cae3-9501-4a08-bfc1-1de676a091d4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.807193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157cae3-9501-4a08-bfc1-1de676a091d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.827203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d157cae3-9501-4a08-bfc1-1de676a091d4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:05.828413 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d157cae3-9501-4a08-bfc1-1de676a091d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:05.871303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0447ad-f9ae-49e5-a5d2-cad3ed47e3c5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:05.898059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:05.911217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 184/358] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:05.911904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157cae3-9501-4a08-bfc1-1de676a091d4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:05.930246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09a395-f999-49d8-977e-af71ad228156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:05.931138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09a395-f999-49d8-977e-af71ad228156 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:05.931899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09a395-f999-49d8-977e-af71ad228156 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:05.932192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09a395-f999-49d8-977e-af71ad228156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:05.933347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09a395-f999-49d8-977e-af71ad228156 None admin] RBAC: Adding query filter params (role.id=6014fae253be445d9675cf76770c7b4c, user.id=ceffc35d267c4cca95dac1e358752f39, scope.system=all) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:05.933673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09a395-f999-49d8-977e-af71ad228156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:05.937972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09a395-f999-49d8-977e-af71ad228156 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:05.958038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 175/359] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 19 12:02:05 2022] GET /identity/v3/role_assignments?user.id=ceffc35d267c4cca95dac1e358752f39&scope.system=all&role.id=6014fae253be445d9675cf76770c7b4c => generated 510 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:05.958742 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09a395-f999-49d8-977e-af71ad228156 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:06.066220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157cae3-9501-4a08-bfc1-1de676a091d4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:06.101872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:06.109025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09a395-f999-49d8-977e-af71ad228156 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:06.125597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac37404-8b91-4c83-a332-8a45de384e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:06.126202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac37404-8b91-4c83-a332-8a45de384e68 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:06.126568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac37404-8b91-4c83-a332-8a45de384e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:06.126853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac37404-8b91-4c83-a332-8a45de384e68 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:06.127620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac37404-8b91-4c83-a332-8a45de384e68 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:06.127951 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac37404-8b91-4c83-a332-8a45de384e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:06.129356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac37404-8b91-4c83-a332-8a45de384e68 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:06.146437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 185/360] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:02:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:06.146437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5f66b-5dbd-4c2b-9669-b7363adec5fc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:06.146437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5f66b-5dbd-4c2b-9669-b7363adec5fc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:06.146437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5f66b-5dbd-4c2b-9669-b7363adec5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:06.163235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ea3f7-3480-49b9-9930-b34d81dad555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:06.164518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea3f7-3480-49b9-9930-b34d81dad555 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:06.165030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea3f7-3480-49b9-9930-b34d81dad555 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:06.165497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea3f7-3480-49b9-9930-b34d81dad555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:06.166368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ea3f7-3480-49b9-9930-b34d81dad555 None admin] RBAC: Adding query filter params (group.id=01371639ab94405783f72e9cb503877c, role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=adcc2a3589e7498db55bb58c9719de51) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:06.166735 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ea3f7-3480-49b9-9930-b34d81dad555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:06.167884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2ed5f66b-5dbd-4c2b-9669-b7363adec5fc None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:06.168475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ea3f7-3480-49b9-9930-b34d81dad555 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:06.169068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2ed5f66b-5dbd-4c2b-9669-b7363adec5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:06.177288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 176/361] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:05 2022] GET /identity/v3/role_assignments?group.id=01371639ab94405783f72e9cb503877c&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:06.178230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ea3f7-3480-49b9-9930-b34d81dad555 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:06.259512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:06.289457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 186/362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:06.290383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5f66b-5dbd-4c2b-9669-b7363adec5fc None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:06.335313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ea3f7-3480-49b9-9930-b34d81dad555 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:06.390746 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99f1a7-cbb9-4a6d-859b-dc9eab76b934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:06.391953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99f1a7-cbb9-4a6d-859b-dc9eab76b934 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:06.392239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99f1a7-cbb9-4a6d-859b-dc9eab76b934 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:06.392522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99f1a7-cbb9-4a6d-859b-dc9eab76b934 None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:06.393853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99f1a7-cbb9-4a6d-859b-dc9eab76b934 None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:06.395434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99f1a7-cbb9-4a6d-859b-dc9eab76b934 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:06.398793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 177/363] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:06 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:06.399528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99f1a7-cbb9-4a6d-859b-dc9eab76b934 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:06.439827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:06.446129 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5f66b-5dbd-4c2b-9669-b7363adec5fc None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:06.498788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83c6a8-f5fa-4953-b875-76a8f752dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:06.499683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83c6a8-f5fa-4953-b875-76a8f752dd7b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:06.499968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83c6a8-f5fa-4953-b875-76a8f752dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:06.500250 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83c6a8-f5fa-4953-b875-76a8f752dd7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:06.512139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83c6a8-f5fa-4953-b875-76a8f752dd7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bb83c6a8-f5fa-4953-b875-76a8f752dd7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:06.522167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:06.524147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 187/364] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:02:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:06.525704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83c6a8-f5fa-4953-b875-76a8f752dd7b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:06.570577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99f1a7-cbb9-4a6d-859b-dc9eab76b934 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:06.589115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:06.627281 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dda23-8bf7-44d7-a80c-100c2d815977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:06.627982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dda23-8bf7-44d7-a80c-100c2d815977 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:06.628268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dda23-8bf7-44d7-a80c-100c2d815977 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:06.628548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dda23-8bf7-44d7-a80c-100c2d815977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:06.675210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83c6a8-f5fa-4953-b875-76a8f752dd7b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:06.719232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dda23-8bf7-44d7-a80c-100c2d815977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:06.720978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dda23-8bf7-44d7-a80c-100c2d815977 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:06.753173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:06.758720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3d2b7-149e-45c5-9a0d-6158a0021841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:06.759740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3d2b7-149e-45c5-9a0d-6158a0021841 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:06.760191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3d2b7-149e-45c5-9a0d-6158a0021841 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:06.760652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3d2b7-149e-45c5-9a0d-6158a0021841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:06.775400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3d2b7-149e-45c5-9a0d-6158a0021841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e9e3d2b7-149e-45c5-9a0d-6158a0021841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:06.785512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:06.786645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:06.787557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:06.789952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 188/365] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:06.791548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3d2b7-149e-45c5-9a0d-6158a0021841 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:06.858471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 178/366] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:06.858471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dda23-8bf7-44d7-a80c-100c2d815977 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:06.941487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3d2b7-149e-45c5-9a0d-6158a0021841 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:06.943406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:06.991717 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de9ca8-af80-4931-98cf-155d433b86c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:06.992388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de9ca8-af80-4931-98cf-155d433b86c6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:06.992676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de9ca8-af80-4931-98cf-155d433b86c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:06.992960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de9ca8-af80-4931-98cf-155d433b86c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:06.993808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de9ca8-af80-4931-98cf-155d433b86c6 None admin] RBAC: Adding query filter params (group.id=01371639ab94405783f72e9cb503877c, role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=adcc2a3589e7498db55bb58c9719de51) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:06.994128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de9ca8-af80-4931-98cf-155d433b86c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:06.995634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de9ca8-af80-4931-98cf-155d433b86c6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:07.002333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 189/367] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:06 2022] GET /identity/v3/role_assignments?group.id=01371639ab94405783f72e9cb503877c&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:07.003400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de9ca8-af80-4931-98cf-155d433b86c6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:07.039596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dda23-8bf7-44d7-a80c-100c2d815977 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:07.073622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:07.089053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f41ff-296e-4a2d-88dc-9ecee81a9336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:07.089865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f41ff-296e-4a2d-88dc-9ecee81a9336 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:07.090046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f41ff-296e-4a2d-88dc-9ecee81a9336 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:07.090282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f41ff-296e-4a2d-88dc-9ecee81a9336 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:07.091129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f41ff-296e-4a2d-88dc-9ecee81a9336 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:07.091439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f41ff-296e-4a2d-88dc-9ecee81a9336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:07.094259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f41ff-296e-4a2d-88dc-9ecee81a9336 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:07.104217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 179/368] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:07.105091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f41ff-296e-4a2d-88dc-9ecee81a9336 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:07.180745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de9ca8-af80-4931-98cf-155d433b86c6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:07.239402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da072c88-24a6-497d-a44c-fee1ef1873dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:07.240453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da072c88-24a6-497d-a44c-fee1ef1873dd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:07.240954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da072c88-24a6-497d-a44c-fee1ef1873dd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:07.241608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da072c88-24a6-497d-a44c-fee1ef1873dd None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:07.243214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da072c88-24a6-497d-a44c-fee1ef1873dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:07.243767 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da072c88-24a6-497d-a44c-fee1ef1873dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:07.247818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da072c88-24a6-497d-a44c-fee1ef1873dd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:07.257116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f41ff-296e-4a2d-88dc-9ecee81a9336 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:07.267884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 190/369] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:07.268956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da072c88-24a6-497d-a44c-fee1ef1873dd None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:07.307220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:07.314182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaae36-5e15-4cda-8176-610e87179399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:07.315116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaae36-5e15-4cda-8176-610e87179399 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:07.315425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaae36-5e15-4cda-8176-610e87179399 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:07.315875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adaae36-5e15-4cda-8176-610e87179399 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:07.320428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adaae36-5e15-4cda-8176-610e87179399 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:07.320428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adaae36-5e15-4cda-8176-610e87179399 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:07.321231 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 180/370] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:07 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:07.322028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaae36-5e15-4cda-8176-610e87179399 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:07.400102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:07.411584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da072c88-24a6-497d-a44c-fee1ef1873dd None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:07.463692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a843684-9e4a-4f61-bd63-dbe5e386f715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:07.464365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a843684-9e4a-4f61-bd63-dbe5e386f715 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:07.464658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a843684-9e4a-4f61-bd63-dbe5e386f715 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:07.464943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a843684-9e4a-4f61-bd63-dbe5e386f715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:07.465955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adaae36-5e15-4cda-8176-610e87179399 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:07.516992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827ceaa-480f-438a-9b0d-d354b3aef833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:07.518129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827ceaa-480f-438a-9b0d-d354b3aef833 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:07.518696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827ceaa-480f-438a-9b0d-d354b3aef833 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:07.519205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827ceaa-480f-438a-9b0d-d354b3aef833 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:07.564728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a843684-9e4a-4f61-bd63-dbe5e386f715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:07.566243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a843684-9e4a-4f61-bd63-dbe5e386f715 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:07.605189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:07.626076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827ceaa-480f-438a-9b0d-d354b3aef833 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:07.628303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827ceaa-480f-438a-9b0d-d354b3aef833 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:07.704139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:07.705359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 191/371] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:07.706707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a843684-9e4a-4f61-bd63-dbe5e386f715 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:07.770390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 181/372] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:07.771695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827ceaa-480f-438a-9b0d-d354b3aef833 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:07.835377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a843684-9e4a-4f61-bd63-dbe5e386f715 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:07.879343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:07.887853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f14a3-23b3-4e43-a4aa-4abbeabf4f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:07.889130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f14a3-23b3-4e43-a4aa-4abbeabf4f91 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:07.889692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f14a3-23b3-4e43-a4aa-4abbeabf4f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:07.890118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f14a3-23b3-4e43-a4aa-4abbeabf4f91 None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:07.892642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f14a3-23b3-4e43-a4aa-4abbeabf4f91 None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:07.893328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f14a3-23b3-4e43-a4aa-4abbeabf4f91 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:07.895161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 192/373] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:07 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:07.896138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f14a3-23b3-4e43-a4aa-4abbeabf4f91 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:07.898136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827ceaa-480f-438a-9b0d-d354b3aef833 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:07.944221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275212ef-66b3-4b0c-9959-19d8897d6e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:07.945262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275212ef-66b3-4b0c-9959-19d8897d6e25 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:07.945804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275212ef-66b3-4b0c-9959-19d8897d6e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:07.945984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275212ef-66b3-4b0c-9959-19d8897d6e25 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:07.947529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275212ef-66b3-4b0c-9959-19d8897d6e25 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:07.949081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275212ef-66b3-4b0c-9959-19d8897d6e25 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:07.958767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 182/374] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:07 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:07.960417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275212ef-66b3-4b0c-9959-19d8897d6e25 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:07.996414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:08.010754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f14a3-23b3-4e43-a4aa-4abbeabf4f91 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:08.059858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724e0af-bf94-46ec-a124-d1be21ff65f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:08.061666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e0af-bf94-46ec-a124-d1be21ff65f0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:08.062159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e0af-bf94-46ec-a124-d1be21ff65f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:08.062683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724e0af-bf94-46ec-a124-d1be21ff65f0 None admin] PATH_INFO: `/v3/projects/67ffb05dd43a48979db84b48835b3e07` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:08.064496 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724e0af-bf94-46ec-a124-d1be21ff65f0 None admin] RBAC: Authorizing `identity:get_project(project_id=67ffb05dd43a48979db84b48835b3e07)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:08.066217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724e0af-bf94-46ec-a124-d1be21ff65f0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:08.068240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 193/375] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:07 2022] GET /identity/v3/projects/67ffb05dd43a48979db84b48835b3e07 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:08.069268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724e0af-bf94-46ec-a124-d1be21ff65f0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:08.089089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275212ef-66b3-4b0c-9959-19d8897d6e25 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:08.127650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec1d3a-378b-4ec2-a75d-3aaaa12d1f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:08.128327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec1d3a-378b-4ec2-a75d-3aaaa12d1f7d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:08.128960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec1d3a-378b-4ec2-a75d-3aaaa12d1f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:08.128960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec1d3a-378b-4ec2-a75d-3aaaa12d1f7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:08.129773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec1d3a-378b-4ec2-a75d-3aaaa12d1f7d None admin] RBAC: Adding query filter params (group.id=01371639ab94405783f72e9cb503877c, role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=f00dbca6b5e149df98838a84028da079) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:08.130087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec1d3a-378b-4ec2-a75d-3aaaa12d1f7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:08.131555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec1d3a-378b-4ec2-a75d-3aaaa12d1f7d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:08.137968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 183/376] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:07 2022] GET /identity/v3/role_assignments?group.id=01371639ab94405783f72e9cb503877c&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:08.138890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec1d3a-378b-4ec2-a75d-3aaaa12d1f7d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:08.140591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:08.188053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724e0af-bf94-46ec-a124-d1be21ff65f0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:08.227021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e508ae-c5c3-4bc1-a7ff-08cf22febb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:08.228262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e508ae-c5c3-4bc1-a7ff-08cf22febb31 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:08.228751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e508ae-c5c3-4bc1-a7ff-08cf22febb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:08.229251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e508ae-c5c3-4bc1-a7ff-08cf22febb31 None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:08.230757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e508ae-c5c3-4bc1-a7ff-08cf22febb31 None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:08.233012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e508ae-c5c3-4bc1-a7ff-08cf22febb31 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:08.235295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 194/377] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:08 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:08.246633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e508ae-c5c3-4bc1-a7ff-08cf22febb31 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:08.251766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:08.282646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec1d3a-378b-4ec2-a75d-3aaaa12d1f7d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:08.352016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3961907-0b1f-42e5-8308-0482885a1cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:08.352939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3961907-0b1f-42e5-8308-0482885a1cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:08.353223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3961907-0b1f-42e5-8308-0482885a1cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:08.353494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3961907-0b1f-42e5-8308-0482885a1cc0 None admin] PATH_INFO: `/v3/groups/8fe798c8eabe4cc68232f2bb6e7516cb` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:08.363485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3961907-0b1f-42e5-8308-0482885a1cc0 None admin] RBAC: Authorizing `identity:get_group(group_id=8fe798c8eabe4cc68232f2bb6e7516cb)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:08.365954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3961907-0b1f-42e5-8308-0482885a1cc0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:08.368062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 184/378] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:08 2022] GET /identity/v3/groups/8fe798c8eabe4cc68232f2bb6e7516cb => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:08.374800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3961907-0b1f-42e5-8308-0482885a1cc0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:08.459853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e508ae-c5c3-4bc1-a7ff-08cf22febb31 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:08.506899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:08.531181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b89a7-c63d-4470-bd63-82a7fca2e913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:08.531871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b89a7-c63d-4470-bd63-82a7fca2e913 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:08.532160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b89a7-c63d-4470-bd63-82a7fca2e913 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:08.532441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b89a7-c63d-4470-bd63-82a7fca2e913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:08.533566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b89a7-c63d-4470-bd63-82a7fca2e913 None admin] RBAC: Adding query filter params (group.id=01371639ab94405783f72e9cb503877c, role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=f00dbca6b5e149df98838a84028da079) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:08.533687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b89a7-c63d-4470-bd63-82a7fca2e913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:08.535100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b89a7-c63d-4470-bd63-82a7fca2e913 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:08.556730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 195/379] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:08 2022] GET /identity/v3/role_assignments?group.id=01371639ab94405783f72e9cb503877c&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 260 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:08.564444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3961907-0b1f-42e5-8308-0482885a1cc0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:08.609907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204b2bf-4524-417d-9be5-8debd0e1cf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:08.611021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204b2bf-4524-417d-9be5-8debd0e1cf89 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:08.611517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204b2bf-4524-417d-9be5-8debd0e1cf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:08.611998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204b2bf-4524-417d-9be5-8debd0e1cf89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:08.613056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204b2bf-4524-417d-9be5-8debd0e1cf89 None admin] RBAC: Adding query filter params (group.id=8fe798c8eabe4cc68232f2bb6e7516cb, role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=67ffb05dd43a48979db84b48835b3e07) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:08.613610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204b2bf-4524-417d-9be5-8debd0e1cf89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:08.615314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204b2bf-4524-417d-9be5-8debd0e1cf89 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:08.621887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 185/380] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:08 2022] GET /identity/v3/role_assignments?group.id=8fe798c8eabe4cc68232f2bb6e7516cb&scope.project.id=67ffb05dd43a48979db84b48835b3e07&role.id=bd295cb5d1194b469beff0381c6b23af => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:08.859141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6d1e8-ab09-4b8f-8a52-8327b3b893ce None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:08.859660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6d1e8-ab09-4b8f-8a52-8327b3b893ce None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:08.859742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6d1e8-ab09-4b8f-8a52-8327b3b893ce None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:08.861461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 196/381] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:08.867214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a3e46-b57e-4559-b367-8b6c31536511 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:08.867713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a3e46-b57e-4559-b367-8b6c31536511 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:08.867786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a3e46-b57e-4559-b367-8b6c31536511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:08.887366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4c8a3e46-b57e-4559-b367-8b6c31536511 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:08.888593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4c8a3e46-b57e-4559-b367-8b6c31536511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:08.900608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:08.972951 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 186/382] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:09.008687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6d1e8-ab09-4b8f-8a52-8327b3b893ce None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:09.190430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6d1e8-ab09-4b8f-8a52-8327b3b893ce None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:09.235431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4206578-9f0c-4f71-b09f-97e19f49cf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:09.236276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4206578-9f0c-4f71-b09f-97e19f49cf9d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:09.236609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4206578-9f0c-4f71-b09f-97e19f49cf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:09.236899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4206578-9f0c-4f71-b09f-97e19f49cf9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:09.259621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4206578-9f0c-4f71-b09f-97e19f49cf9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d4206578-9f0c-4f71-b09f-97e19f49cf9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:09.269019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:09.270016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:09.270851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 197/383] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:09.276048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a3e46-b57e-4559-b367-8b6c31536511 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:09.435035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a3e46-b57e-4559-b367-8b6c31536511 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:09.484798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f624d8-0181-46f0-a993-12b7771e6ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:09.485862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f624d8-0181-46f0-a993-12b7771e6ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:09.486347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f624d8-0181-46f0-a993-12b7771e6ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:09.486866 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f624d8-0181-46f0-a993-12b7771e6ef9 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:09.487867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f624d8-0181-46f0-a993-12b7771e6ef9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:09.488390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f624d8-0181-46f0-a993-12b7771e6ef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:09.490064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f624d8-0181-46f0-a993-12b7771e6ef9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:09.503982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 187/384] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:02:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:09.509961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4206578-9f0c-4f71-b09f-97e19f49cf9d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:09.593031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:09.660126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4206578-9f0c-4f71-b09f-97e19f49cf9d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:09.708027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab976a1-6b7d-4164-900c-257315c3114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:09.708989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab976a1-6b7d-4164-900c-257315c3114f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:09.709272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab976a1-6b7d-4164-900c-257315c3114f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:09.709549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab976a1-6b7d-4164-900c-257315c3114f None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:09.710777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab976a1-6b7d-4164-900c-257315c3114f None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:09.712280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab976a1-6b7d-4164-900c-257315c3114f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:09.714259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 198/385] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:09 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:09.719907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f624d8-0181-46f0-a993-12b7771e6ef9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:09.735173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:09.781411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6025-0f5a-41f5-845c-656a7c59816a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:09.781987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6025-0f5a-41f5-845c-656a7c59816a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:09.782529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c6025-0f5a-41f5-845c-656a7c59816a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:09.784373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 199/386] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:09.789414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be984f-47ac-4b0b-9a32-1ee404a4e15c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:09.789979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be984f-47ac-4b0b-9a32-1ee404a4e15c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:09.790483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be984f-47ac-4b0b-9a32-1ee404a4e15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:09.821971 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c4be984f-47ac-4b0b-9a32-1ee404a4e15c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:09.823229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c4be984f-47ac-4b0b-9a32-1ee404a4e15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:09.899523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f624d8-0181-46f0-a993-12b7771e6ef9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:09.968455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9998f6-ea10-4c45-925a-52eee18ee81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:09.969376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9998f6-ea10-4c45-925a-52eee18ee81a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:09.969662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9998f6-ea10-4c45-925a-52eee18ee81a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:09.969934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9998f6-ea10-4c45-925a-52eee18ee81a None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:09.971165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9998f6-ea10-4c45-925a-52eee18ee81a None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:09.972603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9998f6-ea10-4c45-925a-52eee18ee81a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:09.972782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 200/387] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:09.981424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 188/388] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:09 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:09.987739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be984f-47ac-4b0b-9a32-1ee404a4e15c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:10.001068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9998f6-ea10-4c45-925a-52eee18ee81a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:10.099221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:10.137762 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:10.150254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be984f-47ac-4b0b-9a32-1ee404a4e15c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:10.186383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9998f6-ea10-4c45-925a-52eee18ee81a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:10.190382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1939a0-9cf6-4a7d-bb2c-13d65fa4c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:10.191421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1939a0-9cf6-4a7d-bb2c-13d65fa4c076 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:10.191759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1939a0-9cf6-4a7d-bb2c-13d65fa4c076 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:10.192045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1939a0-9cf6-4a7d-bb2c-13d65fa4c076 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51/groups/01371639ab94405783f72e9cb503877c/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:10.194531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1939a0-9cf6-4a7d-bb2c-13d65fa4c076 None admin] RBAC: Authorizing `identity:create_grant(project_id=adcc2a3589e7498db55bb58c9719de51, group_id=01371639ab94405783f72e9cb503877c, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:10.196097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1939a0-9cf6-4a7d-bb2c-13d65fa4c076 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:10.234549 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfca27-539d-43a5-8c44-60824856162e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:10.235419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfca27-539d-43a5-8c44-60824856162e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:10.235707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfca27-539d-43a5-8c44-60824856162e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:10.235984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfca27-539d-43a5-8c44-60824856162e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:10.247624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfca27-539d-43a5-8c44-60824856162e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1bbfca27-539d-43a5-8c44-60824856162e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:10.257120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:10.266183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 189/389] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:02:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:10.267043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfca27-539d-43a5-8c44-60824856162e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:10.363106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 201/390] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 12:02:09 2022] PUT /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51/groups/01371639ab94405783f72e9cb503877c/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:02:10.372655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfca27-539d-43a5-8c44-60824856162e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:10.416676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921f42b-89e1-443e-81ab-d1db98b606a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:10.417642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921f42b-89e1-443e-81ab-d1db98b606a1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:10.417931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921f42b-89e1-443e-81ab-d1db98b606a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:10.418212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921f42b-89e1-443e-81ab-d1db98b606a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:10.419029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921f42b-89e1-443e-81ab-d1db98b606a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:10.419348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921f42b-89e1-443e-81ab-d1db98b606a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:10.420903 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921f42b-89e1-443e-81ab-d1db98b606a1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:10.422606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:10.435761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 190/391] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:10.441958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1939a0-9cf6-4a7d-bb2c-13d65fa4c076 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:10.593089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1939a0-9cf6-4a7d-bb2c-13d65fa4c076 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:10.609245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:10.632697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc70d72-62bf-487e-a3e1-4cffb7562056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:10.633816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70d72-62bf-487e-a3e1-4cffb7562056 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:10.634245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70d72-62bf-487e-a3e1-4cffb7562056 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:10.634695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc70d72-62bf-487e-a3e1-4cffb7562056 None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:10.643130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc70d72-62bf-487e-a3e1-4cffb7562056 None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:10.643130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc70d72-62bf-487e-a3e1-4cffb7562056 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:10.643130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 202/392] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:10 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:10.646298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921f42b-89e1-443e-81ab-d1db98b606a1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:10.771651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921f42b-89e1-443e-81ab-d1db98b606a1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:10.820851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f7f2d-5efe-469e-a7c1-311e100c8dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:10.821763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f7f2d-5efe-469e-a7c1-311e100c8dda None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:10.822047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f7f2d-5efe-469e-a7c1-311e100c8dda None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:10.822324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f7f2d-5efe-469e-a7c1-311e100c8dda None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:10.827755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f7f2d-5efe-469e-a7c1-311e100c8dda None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:10.829287 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f7f2d-5efe-469e-a7c1-311e100c8dda None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:10.831198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 191/393] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:10 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:10.837583 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc70d72-62bf-487e-a3e1-4cffb7562056 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:11.020242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447f88c-fd78-4a41-a6ae-1992abe49a31 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.020749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447f88c-fd78-4a41-a6ae-1992abe49a31 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.020826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447f88c-fd78-4a41-a6ae-1992abe49a31 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.022689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 192/394] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:11.028071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d93708-752c-4a74-bd8d-dd4e2de2f7b0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.028432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d93708-752c-4a74-bd8d-dd4e2de2f7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.028710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d93708-752c-4a74-bd8d-dd4e2de2f7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.031232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc70d72-62bf-487e-a3e1-4cffb7562056 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:11.055567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-90d93708-752c-4a74-bd8d-dd4e2de2f7b0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:11.056880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-90d93708-752c-4a74-bd8d-dd4e2de2f7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:11.131890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe994f-a67d-4342-90de-a5851d99718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:11.132821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe994f-a67d-4342-90de-a5851d99718c None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.133103 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe994f-a67d-4342-90de-a5851d99718c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.133377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe994f-a67d-4342-90de-a5851d99718c None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51/groups/01371639ab94405783f72e9cb503877c/roles/13607cfcced34c8badf88b8c244d32c5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.135685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe994f-a67d-4342-90de-a5851d99718c None admin] RBAC: Authorizing `identity:create_grant(project_id=adcc2a3589e7498db55bb58c9719de51, group_id=01371639ab94405783f72e9cb503877c, role_id=13607cfcced34c8badf88b8c244d32c5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:11.137143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe994f-a67d-4342-90de-a5851d99718c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:11.170927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:11.177623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 193/395] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:11.201465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d93708-752c-4a74-bd8d-dd4e2de2f7b0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:11.240997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 203/396] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 12:02:10 2022] PUT /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51/groups/01371639ab94405783f72e9cb503877c/roles/13607cfcced34c8badf88b8c244d32c5 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:02:11.368547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d93708-752c-4a74-bd8d-dd4e2de2f7b0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:11.427513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5c3f2-1020-469d-802f-10a175a315d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:11.428342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5c3f2-1020-469d-802f-10a175a315d0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.428629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5c3f2-1020-469d-802f-10a175a315d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.428909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5c3f2-1020-469d-802f-10a175a315d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.448777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5c3f2-1020-469d-802f-10a175a315d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-40a5c3f2-1020-469d-802f-10a175a315d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:11.464719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:11.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:11.467258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:11.468368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 194/397] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:11.479608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe994f-a67d-4342-90de-a5851d99718c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:11.512693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891208b-97b2-4bde-b693-5664dbf40f1d None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.512974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891208b-97b2-4bde-b693-5664dbf40f1d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.513249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891208b-97b2-4bde-b693-5664dbf40f1d None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.515565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 195/398] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:11.521721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de6aa1-5724-4261-9ea3-4133ff74b181 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.522003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de6aa1-5724-4261-9ea3-4133ff74b181 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.522280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de6aa1-5724-4261-9ea3-4133ff74b181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.542272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c8de6aa1-5724-4261-9ea3-4133ff74b181 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:11.544163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c8de6aa1-5724-4261-9ea3-4133ff74b181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:11.564518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:11.614946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:11.645576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 196/399] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:11.650975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a64270-ef94-46d4-815b-bf7fe9ac36f3 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.651295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a64270-ef94-46d4-815b-bf7fe9ac36f3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.651536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a64270-ef94-46d4-815b-bf7fe9ac36f3 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.658445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 197/400] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:11.665816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a20da8-d142-43be-8ef6-c88c399ea6d6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.666165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a20da8-d142-43be-8ef6-c88c399ea6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.666483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a20da8-d142-43be-8ef6-c88c399ea6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.692703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-59a20da8-d142-43be-8ef6-c88c399ea6d6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:11.694205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-59a20da8-d142-43be-8ef6-c88c399ea6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:11.694996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe994f-a67d-4342-90de-a5851d99718c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:11.734448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243b4cc-6531-430d-b658-9c10ec127a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:11.735405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243b4cc-6531-430d-b658-9c10ec127a80 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.735835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243b4cc-6531-430d-b658-9c10ec127a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.736254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243b4cc-6531-430d-b658-9c10ec127a80 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.737150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243b4cc-6531-430d-b658-9c10ec127a80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:11.737606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243b4cc-6531-430d-b658-9c10ec127a80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:11.739385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243b4cc-6531-430d-b658-9c10ec127a80 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:11.749622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 204/401] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:02:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:11.755364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243b4cc-6531-430d-b658-9c10ec127a80 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:11.771655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 198/402] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:11.780052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a20da8-d142-43be-8ef6-c88c399ea6d6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:11.869078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243b4cc-6531-430d-b658-9c10ec127a80 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:11.872474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:11.908341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a20da8-d142-43be-8ef6-c88c399ea6d6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:11.909968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd7bb4-9ebd-4097-b5f6-82ea5a7642fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:11.910805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd7bb4-9ebd-4097-b5f6-82ea5a7642fc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.911091 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd7bb4-9ebd-4097-b5f6-82ea5a7642fc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.911374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd7bb4-9ebd-4097-b5f6-82ea5a7642fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.925263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd7bb4-9ebd-4097-b5f6-82ea5a7642fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:11.925646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d7dd7bb4-9ebd-4097-b5f6-82ea5a7642fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:11.933349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:11.937121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 205/403] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:11.939130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd7bb4-9ebd-4097-b5f6-82ea5a7642fc None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:11.948790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84802736-6ef8-483c-a4e9-f05e671b759b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:11.949667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802736-6ef8-483c-a4e9-f05e671b759b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:11.949955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802736-6ef8-483c-a4e9-f05e671b759b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:11.950230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802736-6ef8-483c-a4e9-f05e671b759b None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:11.952606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802736-6ef8-483c-a4e9-f05e671b759b None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:11.954072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802736-6ef8-483c-a4e9-f05e671b759b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:11.955804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 199/404] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:11 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:11.956588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84802736-6ef8-483c-a4e9-f05e671b759b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:12.088323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd7bb4-9ebd-4097-b5f6-82ea5a7642fc None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:12.105190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84802736-6ef8-483c-a4e9-f05e671b759b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:12.125983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45b7f4-dac4-44fc-9c8b-97020beff7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:12.126898 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45b7f4-dac4-44fc-9c8b-97020beff7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:12.127186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45b7f4-dac4-44fc-9c8b-97020beff7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:12.127468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45b7f4-dac4-44fc-9c8b-97020beff7e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:12.138799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45b7f4-dac4-44fc-9c8b-97020beff7e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-2c45b7f4-dac4-44fc-9c8b-97020beff7e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:12.151962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:12.152926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:12.153694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:12.155202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a45ef-15b8-4f8a-8635-18bccfae958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:12.155202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a45ef-15b8-4f8a-8635-18bccfae958a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:12.155202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a45ef-15b8-4f8a-8635-18bccfae958a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:12.155202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a45ef-15b8-4f8a-8635-18bccfae958a None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:12.155530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a45ef-15b8-4f8a-8635-18bccfae958a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:12.155530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a45ef-15b8-4f8a-8635-18bccfae958a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:12.155657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 206/405] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:02:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:12.156301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45b7f4-dac4-44fc-9c8b-97020beff7e5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:12.156455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a45ef-15b8-4f8a-8635-18bccfae958a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:12.173564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 200/406] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:12.174348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a45ef-15b8-4f8a-8635-18bccfae958a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:12.223935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:12.241924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:12.282802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45b7f4-dac4-44fc-9c8b-97020beff7e5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:12.317293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a45ef-15b8-4f8a-8635-18bccfae958a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:12.322783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565149f4-5d30-4594-973c-396fb801f4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:12.323834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565149f4-5d30-4594-973c-396fb801f4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:12.324131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565149f4-5d30-4594-973c-396fb801f4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:12.324416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565149f4-5d30-4594-973c-396fb801f4d6 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:12.325679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565149f4-5d30-4594-973c-396fb801f4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:12.327579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565149f4-5d30-4594-973c-396fb801f4d6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:12.334395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 207/407] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:12 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:12.335538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565149f4-5d30-4594-973c-396fb801f4d6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:12.361521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d66092-3253-409e-80cd-8aed2b4656df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:12.362680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d66092-3253-409e-80cd-8aed2b4656df None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:12.363217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d66092-3253-409e-80cd-8aed2b4656df None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:12.363672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d66092-3253-409e-80cd-8aed2b4656df None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:12.365105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d66092-3253-409e-80cd-8aed2b4656df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:12.365580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d66092-3253-409e-80cd-8aed2b4656df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:12.367686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d66092-3253-409e-80cd-8aed2b4656df None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:12.385557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 201/408] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:12.386947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d66092-3253-409e-80cd-8aed2b4656df None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:12.460406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:12.472576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565149f4-5d30-4594-973c-396fb801f4d6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:12.509003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d66092-3253-409e-80cd-8aed2b4656df None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:12.518192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:12.518487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7a0d-586e-460e-ad2e-0ff896162d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:12.520032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7a0d-586e-460e-ad2e-0ff896162d87 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:12.520490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7a0d-586e-460e-ad2e-0ff896162d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:12.520915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e7a0d-586e-460e-ad2e-0ff896162d87 None admin] PATH_INFO: `/v3/groups/8fe798c8eabe4cc68232f2bb6e7516cb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:12.523056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7a0d-586e-460e-ad2e-0ff896162d87 None admin] RBAC: Authorizing `identity:get_group(group_id=8fe798c8eabe4cc68232f2bb6e7516cb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:12.524986 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e7a0d-586e-460e-ad2e-0ff896162d87 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:12.527878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 208/409] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:12 2022] GET /identity/v3/groups/8fe798c8eabe4cc68232f2bb6e7516cb => generated 229 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:12.528908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7a0d-586e-460e-ad2e-0ff896162d87 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:12.564782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f4220-0667-409a-90c7-28ff1281501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:12.565692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f4220-0667-409a-90c7-28ff1281501b None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:12.565978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f4220-0667-409a-90c7-28ff1281501b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:12.566252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f4220-0667-409a-90c7-28ff1281501b None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079/groups/01371639ab94405783f72e9cb503877c/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:12.569023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f4220-0667-409a-90c7-28ff1281501b None admin] RBAC: Authorizing `identity:create_grant(project_id=f00dbca6b5e149df98838a84028da079, group_id=01371639ab94405783f72e9cb503877c, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:12.570881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f4220-0667-409a-90c7-28ff1281501b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:12.668974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e7a0d-586e-460e-ad2e-0ff896162d87 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:12.706445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 202/410] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 12:02:12 2022] PUT /identity/v3/projects/f00dbca6b5e149df98838a84028da079/groups/01371639ab94405783f72e9cb503877c/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:02:12.706445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f4220-0667-409a-90c7-28ff1281501b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:12.735676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc3044-98f3-4180-b504-8fdbbae28cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:12.737574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc3044-98f3-4180-b504-8fdbbae28cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:12.738037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc3044-98f3-4180-b504-8fdbbae28cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:12.738329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc3044-98f3-4180-b504-8fdbbae28cb6 None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:12.742053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc3044-98f3-4180-b504-8fdbbae28cb6 None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:12.743876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc3044-98f3-4180-b504-8fdbbae28cb6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:12.749119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 209/411] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:12 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:12.759550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc3044-98f3-4180-b504-8fdbbae28cb6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:12.824027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:12.948675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f4220-0667-409a-90c7-28ff1281501b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:12.961018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc3044-98f3-4180-b504-8fdbbae28cb6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:12.991969 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa473027-222e-4cea-bcf5-c8bb8003a632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:12.993177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa473027-222e-4cea-bcf5-c8bb8003a632 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:12.993660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa473027-222e-4cea-bcf5-c8bb8003a632 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:12.994130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa473027-222e-4cea-bcf5-c8bb8003a632 None admin] PATH_INFO: `/v3/projects/67ffb05dd43a48979db84b48835b3e07` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:12.996449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa473027-222e-4cea-bcf5-c8bb8003a632 None admin] RBAC: Authorizing `identity:get_project(project_id=67ffb05dd43a48979db84b48835b3e07)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:12.999044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa473027-222e-4cea-bcf5-c8bb8003a632 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:13.001220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 203/412] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:12 2022] GET /identity/v3/projects/67ffb05dd43a48979db84b48835b3e07 => generated 344 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:13.007291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa473027-222e-4cea-bcf5-c8bb8003a632 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:13.018777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbc8a5-5875-47f4-b5bb-b97ae7ebb585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:13.020233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbc8a5-5875-47f4-b5bb-b97ae7ebb585 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:13.020734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbc8a5-5875-47f4-b5bb-b97ae7ebb585 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:13.021216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbc8a5-5875-47f4-b5bb-b97ae7ebb585 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:13.026342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbc8a5-5875-47f4-b5bb-b97ae7ebb585 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:13.028243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbc8a5-5875-47f4-b5bb-b97ae7ebb585 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:13.031764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 210/413] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:12 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:13.038063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbc8a5-5875-47f4-b5bb-b97ae7ebb585 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:13.069418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:13.172876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:13.182625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbc8a5-5875-47f4-b5bb-b97ae7ebb585 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:13.220607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa473027-222e-4cea-bcf5-c8bb8003a632 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:13.235895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb403df3-0e4e-44af-afd3-447d428282f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:13.237519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb403df3-0e4e-44af-afd3-447d428282f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:13.238554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb403df3-0e4e-44af-afd3-447d428282f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:13.239016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb403df3-0e4e-44af-afd3-447d428282f5 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079/groups/01371639ab94405783f72e9cb503877c/roles/13607cfcced34c8badf88b8c244d32c5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:13.242865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb403df3-0e4e-44af-afd3-447d428282f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f00dbca6b5e149df98838a84028da079, group_id=01371639ab94405783f72e9cb503877c, role_id=13607cfcced34c8badf88b8c244d32c5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:13.244655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb403df3-0e4e-44af-afd3-447d428282f5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:13.262736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d61205-ee38-44c6-aa6c-104ced17d77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:13.264020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d61205-ee38-44c6-aa6c-104ced17d77e None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:13.269042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d61205-ee38-44c6-aa6c-104ced17d77e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:13.269042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d61205-ee38-44c6-aa6c-104ced17d77e None admin] PATH_INFO: `/v3/projects/67ffb05dd43a48979db84b48835b3e07/groups/8fe798c8eabe4cc68232f2bb6e7516cb/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:13.269042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d61205-ee38-44c6-aa6c-104ced17d77e None admin] RBAC: Authorizing `identity:create_grant(project_id=67ffb05dd43a48979db84b48835b3e07, group_id=8fe798c8eabe4cc68232f2bb6e7516cb, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:13.269376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d61205-ee38-44c6-aa6c-104ced17d77e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:13.385363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 211/414] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 12:02:13 2022] PUT /identity/v3/projects/f00dbca6b5e149df98838a84028da079/groups/01371639ab94405783f72e9cb503877c/roles/13607cfcced34c8badf88b8c244d32c5 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:02:13.387045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d019c6c-266b-4f00-b8a5-d3cc342fad05 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:13.387342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d019c6c-266b-4f00-b8a5-d3cc342fad05 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:13.387624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d019c6c-266b-4f00-b8a5-d3cc342fad05 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:13.394111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 212/415] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:13.400194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bdd03-014e-483e-8813-7be536f05ad4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:13.400585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bdd03-014e-483e-8813-7be536f05ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:13.400938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bdd03-014e-483e-8813-7be536f05ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:13.403250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 204/416] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 19 12:02:13 2022] PUT /identity/v3/projects/67ffb05dd43a48979db84b48835b3e07/groups/8fe798c8eabe4cc68232f2bb6e7516cb/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:02:13.425570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-177bdd03-014e-483e-8813-7be536f05ad4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:13.427039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-177bdd03-014e-483e-8813-7be536f05ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:13.514512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 213/417] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:13.581947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d61205-ee38-44c6-aa6c-104ced17d77e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:13.709740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:13.717387 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d61205-ee38-44c6-aa6c-104ced17d77e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:13.758039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa018b-503d-41ca-8b10-c1f98d0660b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:13.758886 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa018b-503d-41ca-8b10-c1f98d0660b3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:13.759172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa018b-503d-41ca-8b10-c1f98d0660b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:13.759453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa018b-503d-41ca-8b10-c1f98d0660b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:13.773348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa018b-503d-41ca-8b10-c1f98d0660b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-91fa018b-503d-41ca-8b10-c1f98d0660b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:13.783344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:13.784328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:13.785123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:13.785760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 205/418] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:13.791247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bdd03-014e-483e-8813-7be536f05ad4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:13.935970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:13.953466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bdd03-014e-483e-8813-7be536f05ad4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:13.998237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518e6b1-898e-4ee7-badd-2d991abc8113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:13.999341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e6b1-898e-4ee7-badd-2d991abc8113 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:13.999828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e6b1-898e-4ee7-badd-2d991abc8113 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:14.000304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e6b1-898e-4ee7-badd-2d991abc8113 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:14.001306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518e6b1-898e-4ee7-badd-2d991abc8113 None admin] RBAC: Adding query filter params (name=member) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:14.001840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518e6b1-898e-4ee7-badd-2d991abc8113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:14.003492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518e6b1-898e-4ee7-badd-2d991abc8113 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:14.012047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed1f43-63fa-42e8-bb65-cdf0089db687 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:14.012383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed1f43-63fa-42e8-bb65-cdf0089db687 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:14.012616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed1f43-63fa-42e8-bb65-cdf0089db687 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:14.014448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 214/420] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:02:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:14.014584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 206/420] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:14.025568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed1f43-63fa-42e8-bb65-cdf0089db687 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:14.032361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05512ebd-5a03-43c1-9019-09898792f61b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:14.032974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05512ebd-5a03-43c1-9019-09898792f61b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:14.033470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05512ebd-5a03-43c1-9019-09898792f61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:14.074040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-05512ebd-5a03-43c1-9019-09898792f61b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:14.075653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-05512ebd-5a03-43c1-9019-09898792f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:14.169077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 215/421] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 12:02:14.231175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed1f43-63fa-42e8-bb65-cdf0089db687 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:14.238104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05512ebd-5a03-43c1-9019-09898792f61b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:14.332467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383816b-0d06-41b6-b9ec-dfc3af659c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:14.333053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383816b-0d06-41b6-b9ec-dfc3af659c27 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:14.333343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383816b-0d06-41b6-b9ec-dfc3af659c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:14.333631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383816b-0d06-41b6-b9ec-dfc3af659c27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:14.363021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:14.368728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:14.421027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05512ebd-5a03-43c1-9019-09898792f61b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:14.472695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383816b-0d06-41b6-b9ec-dfc3af659c27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:14.474271 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383816b-0d06-41b6-b9ec-dfc3af659c27 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:14.511715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a7647-4095-4baf-abda-e5afe7a97057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:14.512656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a7647-4095-4baf-abda-e5afe7a97057 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:14.512947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a7647-4095-4baf-abda-e5afe7a97057 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:14.514139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a7647-4095-4baf-abda-e5afe7a97057 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:14.543727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a7647-4095-4baf-abda-e5afe7a97057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6b8a7647-4095-4baf-abda-e5afe7a97057 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:14.572594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:14.573546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:14.574435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:14.584633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 216/422] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:02:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:14.590437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a7647-4095-4baf-abda-e5afe7a97057 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:14.834448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 207/423] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 804 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:14.839175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383816b-0d06-41b6-b9ec-dfc3af659c27 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:14.862439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a7647-4095-4baf-abda-e5afe7a97057 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:14.925127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7486ce-4671-4b51-a628-12ba4e8af550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:14.926198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7486ce-4671-4b51-a628-12ba4e8af550 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:14.926198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7486ce-4671-4b51-a628-12ba4e8af550 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:14.926383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7486ce-4671-4b51-a628-12ba4e8af550 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:14.928364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7486ce-4671-4b51-a628-12ba4e8af550 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:14.934440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7486ce-4671-4b51-a628-12ba4e8af550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:14.934440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7486ce-4671-4b51-a628-12ba4e8af550 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:14.942431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:14.950432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 217/424] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:14.957028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7486ce-4671-4b51-a628-12ba4e8af550 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:15.050439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:15.164219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383816b-0d06-41b6-b9ec-dfc3af659c27 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:15.235658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7486ce-4671-4b51-a628-12ba4e8af550 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:15.256494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df90db-f198-4eaa-94b4-a8de487d8da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:15.257372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df90db-f198-4eaa-94b4-a8de487d8da9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:15.257656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df90db-f198-4eaa-94b4-a8de487d8da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:15.257933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df90db-f198-4eaa-94b4-a8de487d8da9 None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:15.263839 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df90db-f198-4eaa-94b4-a8de487d8da9 None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:15.265318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20df90db-f198-4eaa-94b4-a8de487d8da9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:15.267431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 208/425] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:14 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:15.273076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df90db-f198-4eaa-94b4-a8de487d8da9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:15.282367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d7acb-bf70-481c-82d7-d2252e62cb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:15.283772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d7acb-bf70-481c-82d7-d2252e62cb34 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:15.284261 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d7acb-bf70-481c-82d7-d2252e62cb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:15.284767 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d7acb-bf70-481c-82d7-d2252e62cb34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:15.373328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:15.377517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d7acb-bf70-481c-82d7-d2252e62cb34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:15.380861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d7acb-bf70-481c-82d7-d2252e62cb34 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:15.507852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 218/426] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:15.515580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d7acb-bf70-481c-82d7-d2252e62cb34 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:15.557819 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df90db-f198-4eaa-94b4-a8de487d8da9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:15.613549 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:15.630964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe763ad-fa25-4a75-8e46-7f08259f3557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:15.632861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe763ad-fa25-4a75-8e46-7f08259f3557 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:15.633357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe763ad-fa25-4a75-8e46-7f08259f3557 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:15.633864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe763ad-fa25-4a75-8e46-7f08259f3557 None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:15.636066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe763ad-fa25-4a75-8e46-7f08259f3557 None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:15.638843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe763ad-fa25-4a75-8e46-7f08259f3557 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:15.646122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 209/427] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:15 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:15.648495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd864b-84c3-490d-bfdb-728a75f0564e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:15.648778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd864b-84c3-490d-bfdb-728a75f0564e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:15.649055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd864b-84c3-490d-bfdb-728a75f0564e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:15.650623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 210/428] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:15.651332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd864b-84c3-490d-bfdb-728a75f0564e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:15.662355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:15.669051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d7acb-bf70-481c-82d7-d2252e62cb34 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:15.714345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1938479-24ba-44bb-be7b-ff0deacf239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:15.715755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1938479-24ba-44bb-be7b-ff0deacf239e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:15.717100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1938479-24ba-44bb-be7b-ff0deacf239e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:15.717585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1938479-24ba-44bb-be7b-ff0deacf239e None admin] PATH_INFO: `/v3/projects/adcc2a3589e7498db55bb58c9719de51` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:15.719033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1938479-24ba-44bb-be7b-ff0deacf239e None admin] RBAC: Authorizing `identity:get_project(project_id=adcc2a3589e7498db55bb58c9719de51)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:15.721194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1938479-24ba-44bb-be7b-ff0deacf239e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:15.728404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 219/429] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:15 2022] GET /identity/v3/projects/adcc2a3589e7498db55bb58c9719de51 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:15.728404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667816f4-26e9-43f8-828a-e8e995366d3d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:15.728404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667816f4-26e9-43f8-828a-e8e995366d3d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:15.728671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667816f4-26e9-43f8-828a-e8e995366d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:15.749175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-667816f4-26e9-43f8-828a-e8e995366d3d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:15.750497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-667816f4-26e9-43f8-828a-e8e995366d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:15.785469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd864b-84c3-490d-bfdb-728a75f0564e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:15.827471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 220/430] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:15.830749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a65be5-a067-4f64-a5df-c378d18d407d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:15.831628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a65be5-a067-4f64-a5df-c378d18d407d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:15.832236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a65be5-a067-4f64-a5df-c378d18d407d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:15.832679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a65be5-a067-4f64-a5df-c378d18d407d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:15.833681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a65be5-a067-4f64-a5df-c378d18d407d None admin] RBAC: Adding query filter params (group.id=01371639ab94405783f72e9cb503877c, role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=adcc2a3589e7498db55bb58c9719de51) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:15.834161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a65be5-a067-4f64-a5df-c378d18d407d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:15.835805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667816f4-26e9-43f8-828a-e8e995366d3d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:15.836321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a65be5-a067-4f64-a5df-c378d18d407d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:15.850440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 211/431] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:15 2022] GET /identity/v3/role_assignments?group.id=01371639ab94405783f72e9cb503877c&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:15.891532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a65be5-a067-4f64-a5df-c378d18d407d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:16.005273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:16.073013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:16.083008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667816f4-26e9-43f8-828a-e8e995366d3d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:16.098183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a65be5-a067-4f64-a5df-c378d18d407d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:16.147608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f89a9e-d4ba-4431-8b08-3c5151fc7f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:16.147608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f89a9e-d4ba-4431-8b08-3c5151fc7f1d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.147907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f89a9e-d4ba-4431-8b08-3c5151fc7f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.147975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f89a9e-d4ba-4431-8b08-3c5151fc7f1d None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.151206 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f89a9e-d4ba-4431-8b08-3c5151fc7f1d None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:16.152879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f89a9e-d4ba-4431-8b08-3c5151fc7f1d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:16.154792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75304808-f879-4fac-b50c-096c6c6b348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:16.156114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75304808-f879-4fac-b50c-096c6c6b348b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.156114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75304808-f879-4fac-b50c-096c6c6b348b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.156287 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75304808-f879-4fac-b50c-096c6c6b348b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.157455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 221/432] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:15 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:16.167365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f89a9e-d4ba-4431-8b08-3c5151fc7f1d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:16.167984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75304808-f879-4fac-b50c-096c6c6b348b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-75304808-f879-4fac-b50c-096c6c6b348b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:16.178132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:16.179245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 12:02:16.180071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:16.180674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 212/433] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:16.189364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75304808-f879-4fac-b50c-096c6c6b348b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:16.316084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f89a9e-d4ba-4431-8b08-3c5151fc7f1d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:16.318562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75304808-f879-4fac-b50c-096c6c6b348b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:16.334489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:16.357088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394059e9-d74a-4edd-ac2c-411529c9af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:16.358034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394059e9-d74a-4edd-ac2c-411529c9af99 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.358588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394059e9-d74a-4edd-ac2c-411529c9af99 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.359016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394059e9-d74a-4edd-ac2c-411529c9af99 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.359411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092b01c-6261-424f-aec4-88a936e28469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:16.360098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394059e9-d74a-4edd-ac2c-411529c9af99 None admin] RBAC: Adding query filter params (name=member) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:16.360254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092b01c-6261-424f-aec4-88a936e28469 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.360642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092b01c-6261-424f-aec4-88a936e28469 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.360854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394059e9-d74a-4edd-ac2c-411529c9af99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:16.361087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092b01c-6261-424f-aec4-88a936e28469 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.362010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092b01c-6261-424f-aec4-88a936e28469 None admin] RBAC: Adding query filter params (group.id=01371639ab94405783f72e9cb503877c, role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=adcc2a3589e7498db55bb58c9719de51) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:16.362738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092b01c-6261-424f-aec4-88a936e28469 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:16.362884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394059e9-d74a-4edd-ac2c-411529c9af99 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:16.364415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092b01c-6261-424f-aec4-88a936e28469 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:16.373128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 222/434] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:16 2022] GET /identity/v3/role_assignments?group.id=01371639ab94405783f72e9cb503877c&scope.project.id=adcc2a3589e7498db55bb58c9719de51&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:16.373959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 213/435] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:02:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:16.380621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092b01c-6261-424f-aec4-88a936e28469 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:16.418992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:16.434043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62309c8-8e49-45b8-bd2a-29d274c9f028 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.434498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62309c8-8e49-45b8-bd2a-29d274c9f028 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.435564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62309c8-8e49-45b8-bd2a-29d274c9f028 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.437599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 214/436] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:16.443554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49b506-f65f-416c-93b5-0052ee843486 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.444199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49b506-f65f-416c-93b5-0052ee843486 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.444686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49b506-f65f-416c-93b5-0052ee843486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.464829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ba49b506-f65f-416c-93b5-0052ee843486 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:16.466504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ba49b506-f65f-416c-93b5-0052ee843486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:16.521242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092b01c-6261-424f-aec4-88a936e28469 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:16.549140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 215/437] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:16.566429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b78b75-77aa-40ea-9499-0d6a3198de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:16.567138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b78b75-77aa-40ea-9499-0d6a3198de40 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.567428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b78b75-77aa-40ea-9499-0d6a3198de40 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.567703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b78b75-77aa-40ea-9499-0d6a3198de40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.575800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49b506-f65f-416c-93b5-0052ee843486 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:16.645160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b78b75-77aa-40ea-9499-0d6a3198de40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:16.646664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b78b75-77aa-40ea-9499-0d6a3198de40 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:16.679445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:16.690196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49b506-f65f-416c-93b5-0052ee843486 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:16.702290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:16.730604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de484f-d49e-4de7-a651-8aa89b577ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:16.731475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de484f-d49e-4de7-a651-8aa89b577ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.731798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de484f-d49e-4de7-a651-8aa89b577ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.732196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de484f-d49e-4de7-a651-8aa89b577ea2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.743528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de484f-d49e-4de7-a651-8aa89b577ea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-72de484f-d49e-4de7-a651-8aa89b577ea2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:16.752068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:16.752980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:16.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 216/438] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:16.758648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 223/439] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:16.759585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de484f-d49e-4de7-a651-8aa89b577ea2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:16.769195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b78b75-77aa-40ea-9499-0d6a3198de40 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:16.895422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b78b75-77aa-40ea-9499-0d6a3198de40 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:16.897928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de484f-d49e-4de7-a651-8aa89b577ea2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:16.938668 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:16.952272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b044894-1da8-486a-b76a-e88ddb782c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:16.956750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b044894-1da8-486a-b76a-e88ddb782c40 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.956857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6393c0-a898-4145-84e9-966f3799b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:16.956947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b044894-1da8-486a-b76a-e88ddb782c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.956947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b044894-1da8-486a-b76a-e88ddb782c40 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.957084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6393c0-a898-4145-84e9-966f3799b87a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.957084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6393c0-a898-4145-84e9-966f3799b87a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.957084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6393c0-a898-4145-84e9-966f3799b87a None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.957254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b044894-1da8-486a-b76a-e88ddb782c40 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:16.957319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6393c0-a898-4145-84e9-966f3799b87a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:16.957319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6393c0-a898-4145-84e9-966f3799b87a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:16.957445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b044894-1da8-486a-b76a-e88ddb782c40 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:16.957510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6393c0-a898-4145-84e9-966f3799b87a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:16.958192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 224/440] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:16 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:16.959477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb547d0-3b09-42cb-9d95-3230df0f84cb None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.959871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb547d0-3b09-42cb-9d95-3230df0f84cb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.960128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb547d0-3b09-42cb-9d95-3230df0f84cb None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.961678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 225/441] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:16.964507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb547d0-3b09-42cb-9d95-3230df0f84cb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:16.967099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 217/442] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:16.969238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ece39-8f7a-45ef-a022-24c31cc8f744 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:16.969385 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ece39-8f7a-45ef-a022-24c31cc8f744 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:16.969607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ece39-8f7a-45ef-a022-24c31cc8f744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:16.991555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6f2ece39-8f7a-45ef-a022-24c31cc8f744 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:16.994502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6f2ece39-8f7a-45ef-a022-24c31cc8f744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:17.017505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:17.063603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 218/443] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:17.071011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ece39-8f7a-45ef-a022-24c31cc8f744 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:17.087518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb547d0-3b09-42cb-9d95-3230df0f84cb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:17.134484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7a183-de19-4e67-b916-016b2113198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:17.135370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7a183-de19-4e67-b916-016b2113198c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:17.135651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7a183-de19-4e67-b916-016b2113198c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:17.135925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7a183-de19-4e67-b916-016b2113198c None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:17.137215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7a183-de19-4e67-b916-016b2113198c None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:17.139336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7a183-de19-4e67-b916-016b2113198c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:17.141270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 226/444] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:16 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:17.141956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7a183-de19-4e67-b916-016b2113198c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:17.203109 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ece39-8f7a-45ef-a022-24c31cc8f744 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:17.234960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:17.242706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09164eea-6400-4eca-a9f0-8b0451091e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:17.243447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09164eea-6400-4eca-a9f0-8b0451091e90 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:17.247514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09164eea-6400-4eca-a9f0-8b0451091e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:17.247514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09164eea-6400-4eca-a9f0-8b0451091e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:17.276526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7a183-de19-4e67-b916-016b2113198c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:17.288904 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:17.317332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099914e-883b-4b98-bfa3-4f00f129ed1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:17.318180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099914e-883b-4b98-bfa3-4f00f129ed1d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:17.318492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099914e-883b-4b98-bfa3-4f00f129ed1d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:17.318779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099914e-883b-4b98-bfa3-4f00f129ed1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:17.331847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099914e-883b-4b98-bfa3-4f00f129ed1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:17.339184 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09164eea-6400-4eca-a9f0-8b0451091e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f099914e-883b-4b98-bfa3-4f00f129ed1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:17.340127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:17.341173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:17.342531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:17.343848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09164eea-6400-4eca-a9f0-8b0451091e90 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:17.343946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 227/445] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:02:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:17.343946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099914e-883b-4b98-bfa3-4f00f129ed1d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:17.463502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 219/446] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:17.464610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09164eea-6400-4eca-a9f0-8b0451091e90 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:17.476710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099914e-883b-4b98-bfa3-4f00f129ed1d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:17.511867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:17.523561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8752c-1690-413a-a7cb-a7af9f1753ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:17.524700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8752c-1690-413a-a7cb-a7af9f1753ae None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:17.525196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8752c-1690-413a-a7cb-a7af9f1753ae None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:17.525701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8752c-1690-413a-a7cb-a7af9f1753ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:17.527396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8752c-1690-413a-a7cb-a7af9f1753ae None admin] RBAC: Adding query filter params (group.id=01371639ab94405783f72e9cb503877c, role.id=60e46f07f6ab4fdd90ec5d32066e0faa, scope.project.id=f00dbca6b5e149df98838a84028da079) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:17.528070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8752c-1690-413a-a7cb-a7af9f1753ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:17.531122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8752c-1690-413a-a7cb-a7af9f1753ae None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:17.535318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:17.542331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 228/447] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:17 2022] GET /identity/v3/role_assignments?group.id=01371639ab94405783f72e9cb503877c&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=60e46f07f6ab4fdd90ec5d32066e0faa => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:17.544075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8752c-1690-413a-a7cb-a7af9f1753ae None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:17.591988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09164eea-6400-4eca-a9f0-8b0451091e90 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:17.635750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a0ec0-e296-4551-a94b-899438374735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:17.636441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a0ec0-e296-4551-a94b-899438374735 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:17.636724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a0ec0-e296-4551-a94b-899438374735 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:17.637006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a0ec0-e296-4551-a94b-899438374735 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:17.637791 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a0ec0-e296-4551-a94b-899438374735 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:17.638560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a0ec0-e296-4551-a94b-899438374735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:17.639631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a0ec0-e296-4551-a94b-899438374735 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:17.648889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 220/448] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:17.656205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a0ec0-e296-4551-a94b-899438374735 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:17.695878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8752c-1690-413a-a7cb-a7af9f1753ae None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:17.738557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb766088-5060-48f2-bd7f-04821deaa67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:17.739363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb766088-5060-48f2-bd7f-04821deaa67e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:17.739673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb766088-5060-48f2-bd7f-04821deaa67e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:17.740008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb766088-5060-48f2-bd7f-04821deaa67e None admin] PATH_INFO: `/v3/projects/67ffb05dd43a48979db84b48835b3e07` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:17.741300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb766088-5060-48f2-bd7f-04821deaa67e None admin] RBAC: Authorizing `identity:get_project(project_id=67ffb05dd43a48979db84b48835b3e07)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:17.742804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb766088-5060-48f2-bd7f-04821deaa67e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:17.744533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 229/449] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:17 2022] GET /identity/v3/projects/67ffb05dd43a48979db84b48835b3e07 => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:17.750939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb766088-5060-48f2-bd7f-04821deaa67e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:17.801480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a0ec0-e296-4551-a94b-899438374735 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:17.826206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:17.838009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:17.853620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1004d-38da-4c48-bc11-aba118f02bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:17.853620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1004d-38da-4c48-bc11-aba118f02bff None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:17.853620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1004d-38da-4c48-bc11-aba118f02bff None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:17.853620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1004d-38da-4c48-bc11-aba118f02bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:17.895757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb766088-5060-48f2-bd7f-04821deaa67e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:17.942987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057582b5-073a-4268-aec9-cc9b24e92c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:17.943972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057582b5-073a-4268-aec9-cc9b24e92c28 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:17.944263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057582b5-073a-4268-aec9-cc9b24e92c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:17.944545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057582b5-073a-4268-aec9-cc9b24e92c28 None admin] PATH_INFO: `/v3/groups/8fe798c8eabe4cc68232f2bb6e7516cb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:17.946068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057582b5-073a-4268-aec9-cc9b24e92c28 None admin] RBAC: Authorizing `identity:get_group(group_id=8fe798c8eabe4cc68232f2bb6e7516cb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:17.948842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057582b5-073a-4268-aec9-cc9b24e92c28 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:17.951391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 230/450] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:17 2022] GET /identity/v3/groups/8fe798c8eabe4cc68232f2bb6e7516cb => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:17.955043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1004d-38da-4c48-bc11-aba118f02bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:17.957000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1004d-38da-4c48-bc11-aba118f02bff None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:17.958534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057582b5-073a-4268-aec9-cc9b24e92c28 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:18.081336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 221/451] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:18.086009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1004d-38da-4c48-bc11-aba118f02bff None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:18.088532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057582b5-073a-4268-aec9-cc9b24e92c28 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:18.108142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:18.114940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:18.124662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7795aba-888c-4267-be03-a527fd720fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:18.125426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7795aba-888c-4267-be03-a527fd720fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:18.125720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7795aba-888c-4267-be03-a527fd720fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:18.126025 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7795aba-888c-4267-be03-a527fd720fc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:18.126950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7795aba-888c-4267-be03-a527fd720fc0 None admin] RBAC: Adding query filter params (group.id=8fe798c8eabe4cc68232f2bb6e7516cb, role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=67ffb05dd43a48979db84b48835b3e07) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:18.134175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7795aba-888c-4267-be03-a527fd720fc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:18.134175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7795aba-888c-4267-be03-a527fd720fc0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:18.135138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 231/452] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:17 2022] GET /identity/v3/role_assignments?group.id=8fe798c8eabe4cc68232f2bb6e7516cb&scope.project.id=67ffb05dd43a48979db84b48835b3e07&role.id=bd295cb5d1194b469beff0381c6b23af => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:18.193040 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1004d-38da-4c48-bc11-aba118f02bff None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:18.231339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2131c-02bd-4f89-b343-d6497c5d4e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:18.232843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2131c-02bd-4f89-b343-d6497c5d4e25 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:18.233123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2131c-02bd-4f89-b343-d6497c5d4e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:18.233396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2131c-02bd-4f89-b343-d6497c5d4e25 None admin] PATH_INFO: `/v3/projects/f00dbca6b5e149df98838a84028da079` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:18.234748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2131c-02bd-4f89-b343-d6497c5d4e25 None admin] RBAC: Authorizing `identity:get_project(project_id=f00dbca6b5e149df98838a84028da079)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:18.236229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2131c-02bd-4f89-b343-d6497c5d4e25 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:18.237990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 222/453] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:02:18 2022] GET /identity/v3/projects/f00dbca6b5e149df98838a84028da079 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:18.243722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7795aba-888c-4267-be03-a527fd720fc0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:18.352880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7795aba-888c-4267-be03-a527fd720fc0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:18.390742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1a57c-582e-4904-8bca-7e24aeb2cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:18.391651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1a57c-582e-4904-8bca-7e24aeb2cd10 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:18.391933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1a57c-582e-4904-8bca-7e24aeb2cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:18.392209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1a57c-582e-4904-8bca-7e24aeb2cd10 None admin] PATH_INFO: `/v3/groups/01371639ab94405783f72e9cb503877c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:18.393387 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1a57c-582e-4904-8bca-7e24aeb2cd10 None admin] RBAC: Authorizing `identity:get_group(group_id=01371639ab94405783f72e9cb503877c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:18.394866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1a57c-582e-4904-8bca-7e24aeb2cd10 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:18.396580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 232/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 19 12:02:18 2022] GET /identity/v3/groups/01371639ab94405783f72e9cb503877c => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:18.402770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2131c-02bd-4f89-b343-d6497c5d4e25 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:18.490389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:18.502842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2131c-02bd-4f89-b343-d6497c5d4e25 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:18.537211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2318ff-a489-4181-97cf-b9eddb33cf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:18.537862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2318ff-a489-4181-97cf-b9eddb33cf83 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:18.538149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2318ff-a489-4181-97cf-b9eddb33cf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:18.538450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2318ff-a489-4181-97cf-b9eddb33cf83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:18.539284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2318ff-a489-4181-97cf-b9eddb33cf83 None admin] RBAC: Adding query filter params (group.id=01371639ab94405783f72e9cb503877c, role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=f00dbca6b5e149df98838a84028da079) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:18.539601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2318ff-a489-4181-97cf-b9eddb33cf83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:18.541101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2318ff-a489-4181-97cf-b9eddb33cf83 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:18.547317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 223/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 19 12:02:18 2022] GET /identity/v3/role_assignments?group.id=01371639ab94405783f72e9cb503877c&scope.project.id=f00dbca6b5e149df98838a84028da079&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:21.743067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337c62-33eb-483f-be97-16ee08cea08c None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:21.743979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337c62-33eb-483f-be97-16ee08cea08c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:21.744430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337c62-33eb-483f-be97-16ee08cea08c None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:21.746386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 233/456] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:21.760293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a44150-0df9-4671-8c8c-6fc310e3c8fa None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:21.761272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a44150-0df9-4671-8c8c-6fc310e3c8fa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:21.761763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a44150-0df9-4671-8c8c-6fc310e3c8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:21.787681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-36a44150-0df9-4671-8c8c-6fc310e3c8fa None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:21.789398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-36a44150-0df9-4671-8c8c-6fc310e3c8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:21.887580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 224/457] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:21.925650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63337c62-33eb-483f-be97-16ee08cea08c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:21.942833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322a88e-0c3e-424a-9c2e-85fd2252caec None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:21.943474 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322a88e-0c3e-424a-9c2e-85fd2252caec None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:21.943947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322a88e-0c3e-424a-9c2e-85fd2252caec None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:21.945806 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 225/458] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:21.952046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c12571-2d39-493e-92f6-9ef5dce01094 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:21.952618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c12571-2d39-493e-92f6-9ef5dce01094 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:21.953120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c12571-2d39-493e-92f6-9ef5dce01094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:21.976651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-25c12571-2d39-493e-92f6-9ef5dce01094 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:21.979504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-25c12571-2d39-493e-92f6-9ef5dce01094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:22.031337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:22.077093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63337c62-33eb-483f-be97-16ee08cea08c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:22.098437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 226/459] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:22.098437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f74ea-11de-471d-b669-2a8cdba61ddc None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.098437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f74ea-11de-471d-b669-2a8cdba61ddc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.098437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f74ea-11de-471d-b669-2a8cdba61ddc None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.098437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 227/460] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:22.102121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c987cfe-684d-4bbb-b346-b630a961dce7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.102431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c987cfe-684d-4bbb-b346-b630a961dce7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.102861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c987cfe-684d-4bbb-b346-b630a961dce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.124372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6c987cfe-684d-4bbb-b346-b630a961dce7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:22.125700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6c987cfe-684d-4bbb-b346-b630a961dce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:22.135904 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:22.139923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a4304-3b02-4326-92ca-558f59797678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:22.141067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a4304-3b02-4326-92ca-558f59797678 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.141553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a4304-3b02-4326-92ca-558f59797678 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.142030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a4304-3b02-4326-92ca-558f59797678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.156868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a4304-3b02-4326-92ca-558f59797678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:22.158850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a4304-3b02-4326-92ca-558f59797678 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e99a4304-3b02-4326-92ca-558f59797678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:22.179870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:22.180821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:22.181715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:22.181715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:22.181715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:22.181715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:22.184159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 234/461] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:22.185893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a4304-3b02-4326-92ca-558f59797678 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:22.238525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 228/462] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:22.239888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c987cfe-684d-4bbb-b346-b630a961dce7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:22.325251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a4304-3b02-4326-92ca-558f59797678 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:22.337495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:22.357537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c987cfe-684d-4bbb-b346-b630a961dce7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:22.365499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe17c3f-1aa5-4aef-9e31-72d4d238928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:22.366398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe17c3f-1aa5-4aef-9e31-72d4d238928a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.366887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe17c3f-1aa5-4aef-9e31-72d4d238928a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.367316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe17c3f-1aa5-4aef-9e31-72d4d238928a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.379212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe17c3f-1aa5-4aef-9e31-72d4d238928a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:22.381061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe17c3f-1aa5-4aef-9e31-72d4d238928a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:22.396922 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabdb4b-89f3-45a2-8318-fdb50930178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:22.397595 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabdb4b-89f3-45a2-8318-fdb50930178d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.397882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabdb4b-89f3-45a2-8318-fdb50930178d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.398162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabdb4b-89f3-45a2-8318-fdb50930178d None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.398932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabdb4b-89f3-45a2-8318-fdb50930178d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:22.399247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabdb4b-89f3-45a2-8318-fdb50930178d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-4fe17c3f-1aa5-4aef-9e31-72d4d238928a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:22.399831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:22.400651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:22.401482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:22.401482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:22.401482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:22.401482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:22.401706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabdb4b-89f3-45a2-8318-fdb50930178d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:22.406616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 235/463] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:22.407564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe17c3f-1aa5-4aef-9e31-72d4d238928a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:22.418912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 229/464] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:22.418912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332703-4a2e-439d-9ea1-23ce329ace7c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.418912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332703-4a2e-439d-9ea1-23ce329ace7c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.418912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332703-4a2e-439d-9ea1-23ce329ace7c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.422605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 230/465] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:22.424574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01332703-4a2e-439d-9ea1-23ce329ace7c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:22.427802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:22.530007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe17c3f-1aa5-4aef-9e31-72d4d238928a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:22.542389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01332703-4a2e-439d-9ea1-23ce329ace7c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:22.573588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf3a38-431b-40cc-b98a-598d788fa4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:22.574422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf3a38-431b-40cc-b98a-598d788fa4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.574726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf3a38-431b-40cc-b98a-598d788fa4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.575007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf3a38-431b-40cc-b98a-598d788fa4e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.587497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf3a38-431b-40cc-b98a-598d788fa4e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:22.588898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5c6e1-da9a-472f-848d-a6792d76fe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:22.589038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf3a38-431b-40cc-b98a-598d788fa4e0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:22.590021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5c6e1-da9a-472f-848d-a6792d76fe20 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.590662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5c6e1-da9a-472f-848d-a6792d76fe20 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.591711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5c6e1-da9a-472f-848d-a6792d76fe20 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.593055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5c6e1-da9a-472f-848d-a6792d76fe20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:22.593618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5c6e1-da9a-472f-848d-a6792d76fe20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:22.596712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5c6e1-da9a-472f-848d-a6792d76fe20 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-55cf3a38-431b-40cc-b98a-598d788fa4e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:22.608640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:22.609552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:22.611253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:22.611253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:22.611253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:22.611253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:22.611253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 236/466] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:22.611253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf3a38-431b-40cc-b98a-598d788fa4e0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:22.625967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 231/467] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:22.625967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a99f1c-0603-4413-aeef-c5ba44d045df None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.625967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a99f1c-0603-4413-aeef-c5ba44d045df None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.625967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a99f1c-0603-4413-aeef-c5ba44d045df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.635096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:22.641087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f9a99f1c-0603-4413-aeef-c5ba44d045df None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:22.642288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f9a99f1c-0603-4413-aeef-c5ba44d045df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:22.703724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:22.714091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 232/468] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:22.715070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a99f1c-0603-4413-aeef-c5ba44d045df None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:22.724311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf3a38-431b-40cc-b98a-598d788fa4e0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:22.765723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49956f21-498c-4a0a-8a1a-4ee2842c8b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:22.766675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49956f21-498c-4a0a-8a1a-4ee2842c8b86 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.766854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49956f21-498c-4a0a-8a1a-4ee2842c8b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.767142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49956f21-498c-4a0a-8a1a-4ee2842c8b86 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.767879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49956f21-498c-4a0a-8a1a-4ee2842c8b86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:22.769392 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49956f21-498c-4a0a-8a1a-4ee2842c8b86 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:22.821992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a99f1c-0603-4413-aeef-c5ba44d045df None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:22.857091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb7654-3b98-41a3-a962-0ec3a71ca3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:22.857744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb7654-3b98-41a3-a962-0ec3a71ca3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.857937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb7654-3b98-41a3-a962-0ec3a71ca3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.858216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb7654-3b98-41a3-a962-0ec3a71ca3f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.858983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb7654-3b98-41a3-a962-0ec3a71ca3f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:22.859291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb7654-3b98-41a3-a962-0ec3a71ca3f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:22.860680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb7654-3b98-41a3-a962-0ec3a71ca3f7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:22.877081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 237/469] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:02:22 2022] POST /identity/v3/users => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:02:22.877081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dc448-f718-4ea2-a33e-2589fdea4346 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:22.877081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dc448-f718-4ea2-a33e-2589fdea4346 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:22.877081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dc448-f718-4ea2-a33e-2589fdea4346 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:22.877081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 238/470] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:22.877081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dc448-f718-4ea2-a33e-2589fdea4346 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:22.877655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 233/471] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:22.877655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb7654-3b98-41a3-a962-0ec3a71ca3f7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:22.956195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:22.962265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:22.993969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dc448-f718-4ea2-a33e-2589fdea4346 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:23.036955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19ee7a-df60-45fe-837c-55f013868539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:23.047698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19ee7a-df60-45fe-837c-55f013868539 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:23.047698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19ee7a-df60-45fe-837c-55f013868539 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:23.047698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19ee7a-df60-45fe-837c-55f013868539 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:23.047698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19ee7a-df60-45fe-837c-55f013868539 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:23.047698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19ee7a-df60-45fe-837c-55f013868539 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:23.048164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb7654-3b98-41a3-a962-0ec3a71ca3f7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:23.087292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bb7f3-4990-4bb5-83bf-7e1047997a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:23.088214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bb7f3-4990-4bb5-83bf-7e1047997a7a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:23.088491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bb7f3-4990-4bb5-83bf-7e1047997a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:23.088759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bb7f3-4990-4bb5-83bf-7e1047997a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:23.101691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bb7f3-4990-4bb5-83bf-7e1047997a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:23.103182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bb7f3-4990-4bb5-83bf-7e1047997a7a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-926bb7f3-4990-4bb5-83bf-7e1047997a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:23.121467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:23.123563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:23.124610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:23.124610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:23.124610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:23.124610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:23.124610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 234/472] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:23.125798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c843f-d600-48ce-a058-825d3e5ebd61 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:23.125955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c843f-d600-48ce-a058-825d3e5ebd61 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:23.126353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c843f-d600-48ce-a058-825d3e5ebd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:23.132409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 239/473] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:02:22 2022] POST /identity/v3/users => generated 248 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:02:23.134347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19ee7a-df60-45fe-837c-55f013868539 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:23.146389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fe3c843f-d600-48ce-a058-825d3e5ebd61 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:23.148533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fe3c843f-d600-48ce-a058-825d3e5ebd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:23.268124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 235/474] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:23.269249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c843f-d600-48ce-a058-825d3e5ebd61 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:23.285204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19ee7a-df60-45fe-837c-55f013868539 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:23.321579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:23.327391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:23.331538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d459be0-728d-4f11-927c-5c7984c02014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:23.332251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d459be0-728d-4f11-927c-5c7984c02014 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:23.332538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d459be0-728d-4f11-927c-5c7984c02014 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:23.332818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d459be0-728d-4f11-927c-5c7984c02014 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:23.333548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d459be0-728d-4f11-927c-5c7984c02014 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:23.335296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d459be0-728d-4f11-927c-5c7984c02014 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:23.414738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c843f-d600-48ce-a058-825d3e5ebd61 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:23.455974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2255d-ae35-463f-946c-1f7ccafd96a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:23.457036 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2255d-ae35-463f-946c-1f7ccafd96a8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:23.457365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2255d-ae35-463f-946c-1f7ccafd96a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:23.457593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2255d-ae35-463f-946c-1f7ccafd96a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:23.458341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2255d-ae35-463f-946c-1f7ccafd96a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:23.458737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2255d-ae35-463f-946c-1f7ccafd96a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:23.460138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2255d-ae35-463f-946c-1f7ccafd96a8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:23.469966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 240/475] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:02:23 2022] POST /identity/v3/users => generated 248 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:02:23.472081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d459be0-728d-4f11-927c-5c7984c02014 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:23.481253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 236/476] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:23.487593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2255d-ae35-463f-946c-1f7ccafd96a8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:23.656998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d459be0-728d-4f11-927c-5c7984c02014 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:23.733425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:23.742726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2255d-ae35-463f-946c-1f7ccafd96a8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:23.748556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:23.748556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:23.748556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:23.748556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:23.748871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:23.749081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:23.781722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86c93b-4a5a-42a9-bad1-ac48ae88405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:23.782468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86c93b-4a5a-42a9-bad1-ac48ae88405c None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:23.783586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86c93b-4a5a-42a9-bad1-ac48ae88405c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:23.783586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86c93b-4a5a-42a9-bad1-ac48ae88405c None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:23.783762 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86c93b-4a5a-42a9-bad1-ac48ae88405c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:23.785194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86c93b-4a5a-42a9-bad1-ac48ae88405c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:23.798119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.sql.core [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 12:02:23.798119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 12:02:23.798119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [parameters: {'id': 'aeefd82e635040f0ba90d44a67e63561', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 12:02:23.798119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 19 12:02:23.816113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return ref.to_dict() May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 12:02:23.817079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application yield resource May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.rollback() May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.force_reraise() May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise self.value May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application session.commit() May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 12:02:23.817969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._prepare_impl() May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.flush() May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._flush(objects) May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application compat.raise_( May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application flush_context.execute() May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rec.execute(self) May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 12:02:23.818984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 12:02:23.819933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 12:02:23.820721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [parameters: {'id': 'aeefd82e635040f0ba90d44a67e63561', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:23.821482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 12:02:23.822330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:23.823596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 241/477] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 12:02:23 2022] POST /identity/v3/roles => generated 144 bytes in 346 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:23.829045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:23.886779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 237/478] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:02:23 2022] POST /identity/v3/users => generated 249 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:02:23.911741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:24.037058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442510f-d2ae-41eb-b611-64e39d2b4faf None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:24.099003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad0059-ba7e-4837-8695-295c90987660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:24.099853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad0059-ba7e-4837-8695-295c90987660 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:24.100139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad0059-ba7e-4837-8695-295c90987660 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:24.100416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad0059-ba7e-4837-8695-295c90987660 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:24.118325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad0059-ba7e-4837-8695-295c90987660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d7ad0059-ba7e-4837-8695-295c90987660 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:24.128471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:24.129443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:24.130236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 242/479] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:02:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:24.139132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86c93b-4a5a-42a9-bad1-ac48ae88405c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:24.270469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86c93b-4a5a-42a9-bad1-ac48ae88405c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:24.310888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae884d-f9df-480b-b5ef-be2869f7807c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:24.311921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae884d-f9df-480b-b5ef-be2869f7807c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:24.312403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae884d-f9df-480b-b5ef-be2869f7807c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:24.312872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae884d-f9df-480b-b5ef-be2869f7807c None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:24.313830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae884d-f9df-480b-b5ef-be2869f7807c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:24.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae884d-f9df-480b-b5ef-be2869f7807c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:24.315975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae884d-f9df-480b-b5ef-be2869f7807c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:24.327389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 238/480] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:25.814134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e4509-885b-48ec-ab39-2407142d8f44 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:25.814681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e4509-885b-48ec-ab39-2407142d8f44 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:25.814792 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e4509-885b-48ec-ab39-2407142d8f44 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:25.816362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 243/481] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:25.821545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78a18e-5d52-4ede-9904-7ca166e713c4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:25.821984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78a18e-5d52-4ede-9904-7ca166e713c4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:25.822163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78a18e-5d52-4ede-9904-7ca166e713c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:25.844185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ba78a18e-5d52-4ede-9904-7ca166e713c4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:25.845368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ba78a18e-5d52-4ede-9904-7ca166e713c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:25.857780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:25.944290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 239/482] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:25.967928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e4509-885b-48ec-ab39-2407142d8f44 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:26.090376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bae63-267e-4d0d-9c31-1ba9795482b1 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.090726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bae63-267e-4d0d-9c31-1ba9795482b1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.091080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bae63-267e-4d0d-9c31-1ba9795482b1 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.092522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 240/483] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:26.095136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e4509-885b-48ec-ab39-2407142d8f44 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:26.098312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558abd6-c08f-4812-8000-dc9d83dbd933 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.098948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558abd6-c08f-4812-8000-dc9d83dbd933 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.099430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558abd6-c08f-4812-8000-dc9d83dbd933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.108893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:26.138581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b558abd6-c08f-4812-8000-dc9d83dbd933 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:26.141620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b558abd6-c08f-4812-8000-dc9d83dbd933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:26.167260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93e334-2996-4ab3-bbf5-c83afcaff4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:26.168640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93e334-2996-4ab3-bbf5-c83afcaff4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.170645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93e334-2996-4ab3-bbf5-c83afcaff4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.171380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93e334-2996-4ab3-bbf5-c83afcaff4a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.194699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93e334-2996-4ab3-bbf5-c83afcaff4a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-af93e334-2996-4ab3-bbf5-c83afcaff4a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:26.204962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:26.205936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:26.206744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:26.219081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 244/484] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:26.225187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93e334-2996-4ab3-bbf5-c83afcaff4a0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:26.272502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 241/485] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:26.322288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558abd6-c08f-4812-8000-dc9d83dbd933 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:26.389918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93e334-2996-4ab3-bbf5-c83afcaff4a0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:26.430618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:26.436834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d166b0-2b8b-4832-931d-f63976c9e25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:26.438049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d166b0-2b8b-4832-931d-f63976c9e25d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.438545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d166b0-2b8b-4832-931d-f63976c9e25d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.438989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d166b0-2b8b-4832-931d-f63976c9e25d None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.447995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d166b0-2b8b-4832-931d-f63976c9e25d None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:26.447995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d166b0-2b8b-4832-931d-f63976c9e25d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:26.447995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d166b0-2b8b-4832-931d-f63976c9e25d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:26.453309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 245/486] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:26.454964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01905c-4f21-41c0-98f0-2cff42d5ca78 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.455485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01905c-4f21-41c0-98f0-2cff42d5ca78 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.455964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01905c-4f21-41c0-98f0-2cff42d5ca78 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.462523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 246/487] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:26.463668 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01905c-4f21-41c0-98f0-2cff42d5ca78 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:26.475330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558abd6-c08f-4812-8000-dc9d83dbd933 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:26.513840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:26.521542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58616719-cae8-4294-991b-475f153c8333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:26.522468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58616719-cae8-4294-991b-475f153c8333 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.522764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58616719-cae8-4294-991b-475f153c8333 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.523045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58616719-cae8-4294-991b-475f153c8333 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.534483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58616719-cae8-4294-991b-475f153c8333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-58616719-cae8-4294-991b-475f153c8333 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:26.542749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:26.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:26.545065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:26.549491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 242/488] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:26.551027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e0a22-b1a9-46c4-95e7-8bfeefafe89c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.551325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e0a22-b1a9-46c4-95e7-8bfeefafe89c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.551609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e0a22-b1a9-46c4-95e7-8bfeefafe89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.572352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fe4e0a22-b1a9-46c4-95e7-8bfeefafe89c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:26.573564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fe4e0a22-b1a9-46c4-95e7-8bfeefafe89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:26.621167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01905c-4f21-41c0-98f0-2cff42d5ca78 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:26.666466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 243/489] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:26.666466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e0a22-b1a9-46c4-95e7-8bfeefafe89c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:26.687205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919fa1a-8feb-4a1f-beb5-c1d683309d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:26.688023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919fa1a-8feb-4a1f-beb5-c1d683309d89 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.689851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919fa1a-8feb-4a1f-beb5-c1d683309d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.690038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919fa1a-8feb-4a1f-beb5-c1d683309d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.712561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919fa1a-8feb-4a1f-beb5-c1d683309d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:26.718034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919fa1a-8feb-4a1f-beb5-c1d683309d89 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9919fa1a-8feb-4a1f-beb5-c1d683309d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:26.745277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:26.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:26.747078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:26.747078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:26.747078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:26.747078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:26.750332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 247/490] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:26.751655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919fa1a-8feb-4a1f-beb5-c1d683309d89 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:26.774715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:26.821357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e0a22-b1a9-46c4-95e7-8bfeefafe89c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:26.862329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797d649-c8e7-4d21-8f7b-4f766f58431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:26.863767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d649-c8e7-4d21-8f7b-4f766f58431e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.864047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d649-c8e7-4d21-8f7b-4f766f58431e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.864324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d649-c8e7-4d21-8f7b-4f766f58431e None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.865189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797d649-c8e7-4d21-8f7b-4f766f58431e None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:26.865476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797d649-c8e7-4d21-8f7b-4f766f58431e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:26.866913 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797d649-c8e7-4d21-8f7b-4f766f58431e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:26.878013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 244/491] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:26.878740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797d649-c8e7-4d21-8f7b-4f766f58431e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:26.892547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:26.913604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919fa1a-8feb-4a1f-beb5-c1d683309d89 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:26.956829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546bf2a-8bc3-4903-a33c-51366973bbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:26.957545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546bf2a-8bc3-4903-a33c-51366973bbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.957826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546bf2a-8bc3-4903-a33c-51366973bbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.958095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546bf2a-8bc3-4903-a33c-51366973bbb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.970839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546bf2a-8bc3-4903-a33c-51366973bbb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-1546bf2a-8bc3-4903-a33c-51366973bbb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:26.979996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:26.980912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:26.982353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:26.986685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 248/492] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:26.987913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def937ba-ead3-4828-8310-69f9b5119938 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:26.988204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def937ba-ead3-4828-8310-69f9b5119938 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:26.988489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def937ba-ead3-4828-8310-69f9b5119938 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:26.993287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 249/493] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:26.993287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def937ba-ead3-4828-8310-69f9b5119938 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.013701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797d649-c8e7-4d21-8f7b-4f766f58431e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.046080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:27.057276 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6a00a-7a6a-4ca8-a0e7-f4f8d403c5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.057941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6a00a-7a6a-4ca8-a0e7-f4f8d403c5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.058228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6a00a-7a6a-4ca8-a0e7-f4f8d403c5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.058635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6a00a-7a6a-4ca8-a0e7-f4f8d403c5f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.060260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6a00a-7a6a-4ca8-a0e7-f4f8d403c5f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:27.060759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6a00a-7a6a-4ca8-a0e7-f4f8d403c5f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:27.062147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6a00a-7a6a-4ca8-a0e7-f4f8d403c5f0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:27.079294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 245/494] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:27.079294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6a00a-7a6a-4ca8-a0e7-f4f8d403c5f0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.106876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def937ba-ead3-4828-8310-69f9b5119938 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.151049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec566f5-21f0-4451-9364-12364f248d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.152061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec566f5-21f0-4451-9364-12364f248d95 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.152356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec566f5-21f0-4451-9364-12364f248d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.152639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec566f5-21f0-4451-9364-12364f248d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.163817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:27.167203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec566f5-21f0-4451-9364-12364f248d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:27.170599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec566f5-21f0-4451-9364-12364f248d95 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6ec566f5-21f0-4451-9364-12364f248d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:27.191033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:27.192000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:27.192859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:27.192859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:27.192859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:27.192859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:27.192859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 250/495] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:27.194023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13445a-b2ea-4981-a011-16b82d9c89e0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.194023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13445a-b2ea-4981-a011-16b82d9c89e0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.194200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13445a-b2ea-4981-a011-16b82d9c89e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.213230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8f13445a-b2ea-4981-a011-16b82d9c89e0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:27.214352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8f13445a-b2ea-4981-a011-16b82d9c89e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:27.217374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6a00a-7a6a-4ca8-a0e7-f4f8d403c5f0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.256148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01457dff-c7d8-4ac6-aab5-334a4d9e4152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.256827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01457dff-c7d8-4ac6-aab5-334a4d9e4152 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.257116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01457dff-c7d8-4ac6-aab5-334a4d9e4152 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.257398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01457dff-c7d8-4ac6-aab5-334a4d9e4152 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.258170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01457dff-c7d8-4ac6-aab5-334a4d9e4152 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:27.258521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01457dff-c7d8-4ac6-aab5-334a4d9e4152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:27.259917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01457dff-c7d8-4ac6-aab5-334a4d9e4152 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:27.270233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 246/496] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:27.275446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01457dff-c7d8-4ac6-aab5-334a4d9e4152 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.294643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 251/497] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:27.294643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13445a-b2ea-4981-a011-16b82d9c89e0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.335599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:27.395570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01457dff-c7d8-4ac6-aab5-334a4d9e4152 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.420194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13445a-b2ea-4981-a011-16b82d9c89e0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.428122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17520e03-2c63-47e5-bc25-85bad1b5ce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.428913 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17520e03-2c63-47e5-bc25-85bad1b5ce3f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.429194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17520e03-2c63-47e5-bc25-85bad1b5ce3f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.429466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17520e03-2c63-47e5-bc25-85bad1b5ce3f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.437406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17520e03-2c63-47e5-bc25-85bad1b5ce3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:27.438902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17520e03-2c63-47e5-bc25-85bad1b5ce3f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:27.443289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:27.455155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59123f1-fed5-4634-b5f8-4d300e42a249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-17520e03-2c63-47e5-bc25-85bad1b5ce3f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:27.455312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:27.460554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 12:02:27.461599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:27.462100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59123f1-fed5-4634-b5f8-4d300e42a249 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.462100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59123f1-fed5-4634-b5f8-4d300e42a249 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.462100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59123f1-fed5-4634-b5f8-4d300e42a249 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.462273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 247/498] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:27.462339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59123f1-fed5-4634-b5f8-4d300e42a249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:27.462401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17520e03-2c63-47e5-bc25-85bad1b5ce3f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.462504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59123f1-fed5-4634-b5f8-4d300e42a249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:27.462504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59123f1-fed5-4634-b5f8-4d300e42a249 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:27.478899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 252/499] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:27.479678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59123f1-fed5-4634-b5f8-4d300e42a249 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.566263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17520e03-2c63-47e5-bc25-85bad1b5ce3f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.574094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:27.581401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59123f1-fed5-4634-b5f8-4d300e42a249 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.601168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ae16c-1a18-4341-b395-23586477137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.601990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ae16c-1a18-4341-b395-23586477137f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.602321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ae16c-1a18-4341-b395-23586477137f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.602624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ae16c-1a18-4341-b395-23586477137f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.613805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ae16c-1a18-4341-b395-23586477137f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:27.615285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ae16c-1a18-4341-b395-23586477137f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:27.615506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08af3bd-acbb-4cd2-a285-c0ffc85d0d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.616300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08af3bd-acbb-4cd2-a285-c0ffc85d0d0b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.616586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08af3bd-acbb-4cd2-a285-c0ffc85d0d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.616871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08af3bd-acbb-4cd2-a285-c0ffc85d0d0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.627630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08af3bd-acbb-4cd2-a285-c0ffc85d0d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b19ae16c-1a18-4341-b395-23586477137f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:27.633259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:27.634137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:27.635000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:27.635000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:27.635000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:27.635000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:27.635000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 248/500] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d08af3bd-acbb-4cd2-a285-c0ffc85d0d0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:27.635660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:27.636529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:27.637423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:27.638892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55084343-f9c9-4296-ad14-ac6bf6ecef25 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.638892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55084343-f9c9-4296-ad14-ac6bf6ecef25 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.638892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55084343-f9c9-4296-ad14-ac6bf6ecef25 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.639234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 253/501] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:27.639372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 249/502] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:27.639372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55084343-f9c9-4296-ad14-ac6bf6ecef25 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.639927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08af3bd-acbb-4cd2-a285-c0ffc85d0d0b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.685410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:27.742841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08af3bd-acbb-4cd2-a285-c0ffc85d0d0b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.747515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55084343-f9c9-4296-ad14-ac6bf6ecef25 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.776677 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8365f-cdf4-4556-b448-f1ed33322885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.777342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8365f-cdf4-4556-b448-f1ed33322885 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.777621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8365f-cdf4-4556-b448-f1ed33322885 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.777901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8365f-cdf4-4556-b448-f1ed33322885 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.779451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8365f-cdf4-4556-b448-f1ed33322885 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:27.779768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8365f-cdf4-4556-b448-f1ed33322885 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:27.781248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d8365f-cdf4-4556-b448-f1ed33322885 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:27.781755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190891cd-1bc2-4121-8d03-9b23e7482ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.782565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190891cd-1bc2-4121-8d03-9b23e7482ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.782846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190891cd-1bc2-4121-8d03-9b23e7482ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.783119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190891cd-1bc2-4121-8d03-9b23e7482ba7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.790983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190891cd-1bc2-4121-8d03-9b23e7482ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:27.792682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190891cd-1bc2-4121-8d03-9b23e7482ba7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-190891cd-1bc2-4121-8d03-9b23e7482ba7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:27.807203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:27.808279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:27.809210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 250/503] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:27.809954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190891cd-1bc2-4121-8d03-9b23e7482ba7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.817707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 254/504] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:27.819334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495fd37-43db-4f3e-8cfc-3fb6dd393611 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.819633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495fd37-43db-4f3e-8cfc-3fb6dd393611 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.819915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495fd37-43db-4f3e-8cfc-3fb6dd393611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.829925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:27.837577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0495fd37-43db-4f3e-8cfc-3fb6dd393611 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:27.838816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0495fd37-43db-4f3e-8cfc-3fb6dd393611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:27.908795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 255/505] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:27.909414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190891cd-1bc2-4121-8d03-9b23e7482ba7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:27.909746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495fd37-43db-4f3e-8cfc-3fb6dd393611 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:27.941231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:27.953012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca915d9-d9d3-4dc2-a1d4-d25e8a4c70b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:27.953676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca915d9-d9d3-4dc2-a1d4-d25e8a4c70b3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:27.953961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca915d9-d9d3-4dc2-a1d4-d25e8a4c70b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:27.954237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca915d9-d9d3-4dc2-a1d4-d25e8a4c70b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:27.954991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca915d9-d9d3-4dc2-a1d4-d25e8a4c70b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:27.955297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca915d9-d9d3-4dc2-a1d4-d25e8a4c70b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:27.956670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca915d9-d9d3-4dc2-a1d4-d25e8a4c70b3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:27.969005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 251/506] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:27.970078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca915d9-d9d3-4dc2-a1d4-d25e8a4c70b3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.016277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495fd37-43db-4f3e-8cfc-3fb6dd393611 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.052152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ada05-532f-4d3f-b508-019d76c9c12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.052804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ada05-532f-4d3f-b508-019d76c9c12d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.053093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ada05-532f-4d3f-b508-019d76c9c12d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.053259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:28.053492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ada05-532f-4d3f-b508-019d76c9c12d None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.054601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ada05-532f-4d3f-b508-019d76c9c12d None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:28.054722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ada05-532f-4d3f-b508-019d76c9c12d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.056096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ada05-532f-4d3f-b508-019d76c9c12d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.065056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 256/507] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:28.065783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ada05-532f-4d3f-b508-019d76c9c12d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.078440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca915d9-d9d3-4dc2-a1d4-d25e8a4c70b3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.114260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41965a-5aa7-4a00-b9f5-2cf3f570dcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.114959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41965a-5aa7-4a00-b9f5-2cf3f570dcfc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.115246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41965a-5aa7-4a00-b9f5-2cf3f570dcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.115524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41965a-5aa7-4a00-b9f5-2cf3f570dcfc None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.116362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41965a-5aa7-4a00-b9f5-2cf3f570dcfc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:28.116676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41965a-5aa7-4a00-b9f5-2cf3f570dcfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.118107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41965a-5aa7-4a00-b9f5-2cf3f570dcfc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.158742 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 252/508] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 12:02:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:28.159591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41965a-5aa7-4a00-b9f5-2cf3f570dcfc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.167131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:28.176748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ada05-532f-4d3f-b508-019d76c9c12d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.213323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8dd2e-b068-4270-bc00-23e070f06335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.213979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8dd2e-b068-4270-bc00-23e070f06335 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.214260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8dd2e-b068-4270-bc00-23e070f06335 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.214565 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8dd2e-b068-4270-bc00-23e070f06335 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.261697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41965a-5aa7-4a00-b9f5-2cf3f570dcfc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.281567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8dd2e-b068-4270-bc00-23e070f06335 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.283033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8dd2e-b068-4270-bc00-23e070f06335 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.296337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f2c39-a182-49ae-965f-5f2fe9fb518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.297133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f2c39-a182-49ae-965f-5f2fe9fb518f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.297412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f2c39-a182-49ae-965f-5f2fe9fb518f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.297693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f2c39-a182-49ae-965f-5f2fe9fb518f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.300510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:28.308524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f2c39-a182-49ae-965f-5f2fe9fb518f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.310006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f2c39-a182-49ae-965f-5f2fe9fb518f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-443f2c39-a182-49ae-965f-5f2fe9fb518f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:28.327761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:28.328770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:28.329603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:28.329603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:28.329603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:28.329603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:28.329603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 253/509] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:28.329857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f2c39-a182-49ae-965f-5f2fe9fb518f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.376849 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:28.386782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 257/510] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:28.388199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8dd2e-b068-4270-bc00-23e070f06335 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.432177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f2c39-a182-49ae-965f-5f2fe9fb518f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.465916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67beaf-ef31-45a9-8acb-7bac8de7b1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.466737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67beaf-ef31-45a9-8acb-7bac8de7b1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.467025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67beaf-ef31-45a9-8acb-7bac8de7b1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.467305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67beaf-ef31-45a9-8acb-7bac8de7b1b0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.474756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67beaf-ef31-45a9-8acb-7bac8de7b1b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.476212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67beaf-ef31-45a9-8acb-7bac8de7b1b0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.487078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8dd2e-b068-4270-bc00-23e070f06335 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-7b67beaf-ef31-45a9-8acb-7bac8de7b1b0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:28.490918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:28.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:28.492813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 254/511] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:28.493550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67beaf-ef31-45a9-8acb-7bac8de7b1b0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.521736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821bc0-e220-4b6f-a03d-f7ad2660d359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.522565 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821bc0-e220-4b6f-a03d-f7ad2660d359 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.522841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821bc0-e220-4b6f-a03d-f7ad2660d359 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.523120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821bc0-e220-4b6f-a03d-f7ad2660d359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.533875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8821bc0-e220-4b6f-a03d-f7ad2660d359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.535310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8821bc0-e220-4b6f-a03d-f7ad2660d359 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.549901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b8821bc0-e220-4b6f-a03d-f7ad2660d359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:28.553550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:28.554580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:28.555480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:28.555480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:28.555480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:28.555480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:28.555480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 258/512] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:28.555803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821bc0-e220-4b6f-a03d-f7ad2660d359 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.598162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67beaf-ef31-45a9-8acb-7bac8de7b1b0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.602520 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:28.631797 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c89ea5-6208-4450-9cbb-c9b5e9027317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.632455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c89ea5-6208-4450-9cbb-c9b5e9027317 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.632744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c89ea5-6208-4450-9cbb-c9b5e9027317 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.633026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c89ea5-6208-4450-9cbb-c9b5e9027317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.655220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821bc0-e220-4b6f-a03d-f7ad2660d359 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.688267 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9c53-a3b8-4f79-b8e0-40636558f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.688883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9c53-a3b8-4f79-b8e0-40636558f97a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.689167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9c53-a3b8-4f79-b8e0-40636558f97a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.689442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9c53-a3b8-4f79-b8e0-40636558f97a None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.690155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b9c53-a3b8-4f79-b8e0-40636558f97a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:28.690492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b9c53-a3b8-4f79-b8e0-40636558f97a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.691871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b9c53-a3b8-4f79-b8e0-40636558f97a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.701202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c89ea5-6208-4450-9cbb-c9b5e9027317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.702654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c89ea5-6208-4450-9cbb-c9b5e9027317 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.704202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 259/513] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:28.705176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9c53-a3b8-4f79-b8e0-40636558f97a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.755776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:28.807451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9c53-a3b8-4f79-b8e0-40636558f97a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.808693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 255/514] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:28.810154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c89ea5-6208-4450-9cbb-c9b5e9027317 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.815789 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:28.843288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6951bb-7cd7-4c8f-8b7e-7ac6043f5a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.844088 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6951bb-7cd7-4c8f-8b7e-7ac6043f5a7e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.844375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6951bb-7cd7-4c8f-8b7e-7ac6043f5a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.844655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6951bb-7cd7-4c8f-8b7e-7ac6043f5a7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.854970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6951bb-7cd7-4c8f-8b7e-7ac6043f5a7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.857385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6951bb-7cd7-4c8f-8b7e-7ac6043f5a7e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-3b6951bb-7cd7-4c8f-8b7e-7ac6043f5a7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:28.875538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:28.876584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:28.877558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:28.877558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:28.877558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:28.877558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:28.877558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 260/515] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:28.877558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6951bb-7cd7-4c8f-8b7e-7ac6043f5a7e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.912420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c89ea5-6208-4450-9cbb-c9b5e9027317 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:28.947222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681cc53-4598-4ccf-af70-6be033107d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:28.947948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681cc53-4598-4ccf-af70-6be033107d80 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:28.948242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681cc53-4598-4ccf-af70-6be033107d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:28.948551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681cc53-4598-4ccf-af70-6be033107d80 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:28.949395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681cc53-4598-4ccf-af70-6be033107d80 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:28.949712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681cc53-4598-4ccf-af70-6be033107d80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:28.951305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681cc53-4598-4ccf-af70-6be033107d80 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:28.969283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 256/516] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 12:02:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:28.970080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681cc53-4598-4ccf-af70-6be033107d80 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:28.978285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:28.986133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6951bb-7cd7-4c8f-8b7e-7ac6043f5a7e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.021472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a93ae-d8b8-4982-bbc7-381e76073340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.022940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a93ae-d8b8-4982-bbc7-381e76073340 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.023019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a93ae-d8b8-4982-bbc7-381e76073340 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.023257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a93ae-d8b8-4982-bbc7-381e76073340 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.024145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a93ae-d8b8-4982-bbc7-381e76073340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:29.024453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a93ae-d8b8-4982-bbc7-381e76073340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.026001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a93ae-d8b8-4982-bbc7-381e76073340 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.037984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 261/517] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:29.038822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a93ae-d8b8-4982-bbc7-381e76073340 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:29.057906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:29.070684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681cc53-4598-4ccf-af70-6be033107d80 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.107812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d2b6c-5fd6-4790-b66b-f7ae76327bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.108590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2b6c-5fd6-4790-b66b-f7ae76327bf3 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.108870 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2b6c-5fd6-4790-b66b-f7ae76327bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.109143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2b6c-5fd6-4790-b66b-f7ae76327bf3 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.109846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d2b6c-5fd6-4790-b66b-f7ae76327bf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.112110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d2b6c-5fd6-4790-b66b-f7ae76327bf3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.140180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a93ae-d8b8-4982-bbc7-381e76073340 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.174809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c0b4b-bdf3-4a15-9923-b72530c48a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.175595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c0b4b-bdf3-4a15-9923-b72530c48a9d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.175879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c0b4b-bdf3-4a15-9923-b72530c48a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.176156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c0b4b-bdf3-4a15-9923-b72530c48a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.186830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c0b4b-bdf3-4a15-9923-b72530c48a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.188279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c0b4b-bdf3-4a15-9923-b72530c48a9d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-178c0b4b-bdf3-4a15-9923-b72530c48a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:29.205723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:29.206734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:29.207759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:29.207759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:29.207759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:29.207759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:29.207759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 262/518] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:29.207759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c0b4b-bdf3-4a15-9923-b72530c48a9d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:29.226435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 257/519] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:02:28 2022] POST /identity/v3/users => generated 247 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:02:29.227399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d2b6c-5fd6-4790-b66b-f7ae76327bf3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:29.283639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:29.310320 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:29.324497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c0b4b-bdf3-4a15-9923-b72530c48a9d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.354510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d2b6c-5fd6-4790-b66b-f7ae76327bf3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.363690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320be3de-95ee-4aa0-8aaf-0ac4cf1df7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.364397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320be3de-95ee-4aa0-8aaf-0ac4cf1df7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.364723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320be3de-95ee-4aa0-8aaf-0ac4cf1df7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.365006 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320be3de-95ee-4aa0-8aaf-0ac4cf1df7d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.365752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320be3de-95ee-4aa0-8aaf-0ac4cf1df7d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:29.366094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320be3de-95ee-4aa0-8aaf-0ac4cf1df7d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.367518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320be3de-95ee-4aa0-8aaf-0ac4cf1df7d8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.381257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 263/520] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:29.382151 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320be3de-95ee-4aa0-8aaf-0ac4cf1df7d8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:29.399710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aa752-f81c-4357-9a8e-403e5eacc228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.400649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aa752-f81c-4357-9a8e-403e5eacc228 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.400649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aa752-f81c-4357-9a8e-403e5eacc228 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.400911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aa752-f81c-4357-9a8e-403e5eacc228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.486973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92aa752-f81c-4357-9a8e-403e5eacc228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.488548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92aa752-f81c-4357-9a8e-403e5eacc228 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.503235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320be3de-95ee-4aa0-8aaf-0ac4cf1df7d8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.532080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:29.540984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bdc60-db5d-4e18-83af-ca1ea78a7a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.546742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bdc60-db5d-4e18-83af-ca1ea78a7a14 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.546742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bdc60-db5d-4e18-83af-ca1ea78a7a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.546742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bdc60-db5d-4e18-83af-ca1ea78a7a14 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.550290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bdc60-db5d-4e18-83af-ca1ea78a7a14 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.551800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bdc60-db5d-4e18-83af-ca1ea78a7a14 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.561152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-506bdc60-db5d-4e18-83af-ca1ea78a7a14 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:29.569430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:29.570698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 264/521] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:29.571606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bdc60-db5d-4e18-83af-ca1ea78a7a14 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:29.613167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 258/522] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:29.613167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aa752-f81c-4357-9a8e-403e5eacc228 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:29.686446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bdc60-db5d-4e18-83af-ca1ea78a7a14 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.721591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aa752-f81c-4357-9a8e-403e5eacc228 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.723672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a7e0-00e8-4acb-b79c-843c19f820ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.724342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a7e0-00e8-4acb-b79c-843c19f820ac None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.724630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a7e0-00e8-4acb-b79c-843c19f820ac None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.724908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a7e0-00e8-4acb-b79c-843c19f820ac None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.725647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982a7e0-00e8-4acb-b79c-843c19f820ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:29.725960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982a7e0-00e8-4acb-b79c-843c19f820ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.727386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982a7e0-00e8-4acb-b79c-843c19f820ac None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.739672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 265/523] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:29.740513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a7e0-00e8-4acb-b79c-843c19f820ac None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:29.768917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd023e-d9c2-4510-8216-703a7492ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.769983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd023e-d9c2-4510-8216-703a7492ae43 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.770269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd023e-d9c2-4510-8216-703a7492ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.770740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd023e-d9c2-4510-8216-703a7492ae43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.780327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd023e-d9c2-4510-8216-703a7492ae43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.782100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd023e-d9c2-4510-8216-703a7492ae43 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.793030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-fbcd023e-d9c2-4510-8216-703a7492ae43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:29.798217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:29.799303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 259/524] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:29.800315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd023e-d9c2-4510-8216-703a7492ae43 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:29.830048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:29.857232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a7e0-00e8-4acb-b79c-843c19f820ac None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.892581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13a390-cb52-40b2-bcd8-abfb5f7c6e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.893256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13a390-cb52-40b2-bcd8-abfb5f7c6e9c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.893543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13a390-cb52-40b2-bcd8-abfb5f7c6e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.893820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13a390-cb52-40b2-bcd8-abfb5f7c6e9c None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.894686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13a390-cb52-40b2-bcd8-abfb5f7c6e9c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:29.895007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13a390-cb52-40b2-bcd8-abfb5f7c6e9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.896442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13a390-cb52-40b2-bcd8-abfb5f7c6e9c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.911869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd023e-d9c2-4510-8216-703a7492ae43 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:29.920871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 266/525] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:29.920871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13a390-cb52-40b2-bcd8-abfb5f7c6e9c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:29.950998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81456aa5-f3c0-41dc-97ec-4c208019e133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:29.951930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81456aa5-f3c0-41dc-97ec-4c208019e133 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:29.952221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81456aa5-f3c0-41dc-97ec-4c208019e133 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:29.952502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81456aa5-f3c0-41dc-97ec-4c208019e133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:29.963340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81456aa5-f3c0-41dc-97ec-4c208019e133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:29.964816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81456aa5-f3c0-41dc-97ec-4c208019e133 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-81456aa5-f3c0-41dc-97ec-4c208019e133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:29.983021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:29.983934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:29.984766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:29.984766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:29.984766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:29.984766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:29.984766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 260/526] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:29.985039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81456aa5-f3c0-41dc-97ec-4c208019e133 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.039988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13a390-cb52-40b2-bcd8-abfb5f7c6e9c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:30.049224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:30.077893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9a7a6-46c4-4679-94e9-0cf50584d720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:30.078743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9a7a6-46c4-4679-94e9-0cf50584d720 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:30.079032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9a7a6-46c4-4679-94e9-0cf50584d720 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:30.079308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9a7a6-46c4-4679-94e9-0cf50584d720 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:30.080446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:30.086843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9a7a6-46c4-4679-94e9-0cf50584d720 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:30.088301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9a7a6-46c4-4679-94e9-0cf50584d720 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bda9a7a6-46c4-4679-94e9-0cf50584d720 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:30.104503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:30.105429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:30.107233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:30.107233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:30.107233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:30.107233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:30.107233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:30.107233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:30.107233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:30.107233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:30.107749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81456aa5-f3c0-41dc-97ec-4c208019e133 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:30.110994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 267/527] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:30.111926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9a7a6-46c4-4679-94e9-0cf50584d720 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.142982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aec43-14f6-4aea-8852-d16558873391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:30.143770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aec43-14f6-4aea-8852-d16558873391 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:30.144055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aec43-14f6-4aea-8852-d16558873391 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:30.144340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5aec43-14f6-4aea-8852-d16558873391 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:30.145131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5aec43-14f6-4aea-8852-d16558873391 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:30.145452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5aec43-14f6-4aea-8852-d16558873391 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:30.146950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5aec43-14f6-4aea-8852-d16558873391 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:30.165824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 261/528] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:30.166675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aec43-14f6-4aea-8852-d16558873391 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.222379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9a7a6-46c4-4679-94e9-0cf50584d720 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:30.262827 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97116b1-5b7c-4e62-9e73-d2048d0155da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:30.263570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97116b1-5b7c-4e62-9e73-d2048d0155da None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:30.263861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97116b1-5b7c-4e62-9e73-d2048d0155da None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:30.264144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97116b1-5b7c-4e62-9e73-d2048d0155da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:30.275936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5aec43-14f6-4aea-8852-d16558873391 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:30.302556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:30.313326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2700cc-05c0-4283-96a2-9e050d6a9016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:30.313994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2700cc-05c0-4283-96a2-9e050d6a9016 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:30.314283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2700cc-05c0-4283-96a2-9e050d6a9016 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:30.314607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2700cc-05c0-4283-96a2-9e050d6a9016 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:30.315344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2700cc-05c0-4283-96a2-9e050d6a9016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:30.317567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2700cc-05c0-4283-96a2-9e050d6a9016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:30.317567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2700cc-05c0-4283-96a2-9e050d6a9016 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:30.318741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:30.333479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 262/529] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:30.333479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2700cc-05c0-4283-96a2-9e050d6a9016 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.345559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97116b1-5b7c-4e62-9e73-d2048d0155da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:30.352137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97116b1-5b7c-4e62-9e73-d2048d0155da None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:30.435736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2700cc-05c0-4283-96a2-9e050d6a9016 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:30.466350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 268/530] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:30.467520 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97116b1-5b7c-4e62-9e73-d2048d0155da None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.476495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece34e0-bd59-431a-80ee-5741dc8840a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:30.477162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece34e0-bd59-431a-80ee-5741dc8840a5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:30.477450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece34e0-bd59-431a-80ee-5741dc8840a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:30.477735 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece34e0-bd59-431a-80ee-5741dc8840a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:30.478581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece34e0-bd59-431a-80ee-5741dc8840a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:30.478905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece34e0-bd59-431a-80ee-5741dc8840a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:30.480352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece34e0-bd59-431a-80ee-5741dc8840a5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:30.494301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 263/531] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:30.499637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece34e0-bd59-431a-80ee-5741dc8840a5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.537867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:30.566399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:30.576792 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97116b1-5b7c-4e62-9e73-d2048d0155da None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:30.613402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a6f62-608c-43fe-80cd-8f9c3334047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:30.614105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a6f62-608c-43fe-80cd-8f9c3334047d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:30.614320 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece34e0-bd59-431a-80ee-5741dc8840a5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:30.614512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a6f62-608c-43fe-80cd-8f9c3334047d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:30.614700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a6f62-608c-43fe-80cd-8f9c3334047d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:30.615538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a6f62-608c-43fe-80cd-8f9c3334047d None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=9969632e796345fa9671a45cf61ede3e) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:30.615856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a6f62-608c-43fe-80cd-8f9c3334047d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:30.617301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a6f62-608c-43fe-80cd-8f9c3334047d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:30.622998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 269/532] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:30 2022] GET /identity/v3/role_assignments?user.id=9969632e796345fa9671a45cf61ede3e&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:30.623806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a6f62-608c-43fe-80cd-8f9c3334047d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.670580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdaf70-4310-4efa-88a9-d09b3cb639ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:30.671400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdaf70-4310-4efa-88a9-d09b3cb639ce None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:30.671687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdaf70-4310-4efa-88a9-d09b3cb639ce None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:30.671963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdaf70-4310-4efa-88a9-d09b3cb639ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:30.679990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdaf70-4310-4efa-88a9-d09b3cb639ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:30.681632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdaf70-4310-4efa-88a9-d09b3cb639ce None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1cfdaf70-4310-4efa-88a9-d09b3cb639ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:30.697672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:30.698631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:30.699491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:30.699491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:30.699491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:30.699491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:30.699491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:30.699491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:30.699491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:30.699491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:30.704387 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 264/533] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:30.705313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdaf70-4310-4efa-88a9-d09b3cb639ce None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.753058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a6f62-608c-43fe-80cd-8f9c3334047d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:30.794714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:30.801205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eed2c-94d1-44f2-8352-7fc496c12ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:30.802260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eed2c-94d1-44f2-8352-7fc496c12ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:30.802903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eed2c-94d1-44f2-8352-7fc496c12ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:30.803115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eed2c-94d1-44f2-8352-7fc496c12ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:30.815743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eed2c-94d1-44f2-8352-7fc496c12ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:30.862794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eed2c-94d1-44f2-8352-7fc496c12ce7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:30.862930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdaf70-4310-4efa-88a9-d09b3cb639ce None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-b32eed2c-94d1-44f2-8352-7fc496c12ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:30.863010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:30.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:30.865981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:30.865981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:30.865981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:30.865981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:30.865981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 270/534] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:30.865981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eed2c-94d1-44f2-8352-7fc496c12ce7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.866345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:30.868916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d2e8-e9e5-4cde-ab3b-86ab78e59d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:30.869579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d2e8-e9e5-4cde-ab3b-86ab78e59d66 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:30.869866 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d2e8-e9e5-4cde-ab3b-86ab78e59d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:30.870145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d2e8-e9e5-4cde-ab3b-86ab78e59d66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:30.871019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d2e8-e9e5-4cde-ab3b-86ab78e59d66 None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=bb6776311f354b748d563c4f1d793972) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:30.871339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d2e8-e9e5-4cde-ab3b-86ab78e59d66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:30.873011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d2e8-e9e5-4cde-ab3b-86ab78e59d66 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:30.886562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 265/535] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:30 2022] GET /identity/v3/role_assignments?user.id=bb6776311f354b748d563c4f1d793972&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:30.888549 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d2e8-e9e5-4cde-ab3b-86ab78e59d66 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:30.975410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eed2c-94d1-44f2-8352-7fc496c12ce7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:31.017169 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8df87-3936-449b-966e-a8b7ac1b1495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:31.024732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8df87-3936-449b-966e-a8b7ac1b1495 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:31.025308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8df87-3936-449b-966e-a8b7ac1b1495 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:31.025767 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8df87-3936-449b-966e-a8b7ac1b1495 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:31.027079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8df87-3936-449b-966e-a8b7ac1b1495 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:31.027590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8df87-3936-449b-966e-a8b7ac1b1495 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:31.029311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8df87-3936-449b-966e-a8b7ac1b1495 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:31.051868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 271/536] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:31.058385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8df87-3936-449b-966e-a8b7ac1b1495 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:31.139710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d2e8-e9e5-4cde-ab3b-86ab78e59d66 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:31.155690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:31.186442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d879bd9-8566-4f1b-b1b1-9f5257304600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:31.186442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d879bd9-8566-4f1b-b1b1-9f5257304600 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:31.186442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d879bd9-8566-4f1b-b1b1-9f5257304600 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:31.186442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d879bd9-8566-4f1b-b1b1-9f5257304600 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:31.186850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d879bd9-8566-4f1b-b1b1-9f5257304600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:31.187107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d879bd9-8566-4f1b-b1b1-9f5257304600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:31.188499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d879bd9-8566-4f1b-b1b1-9f5257304600 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:31.202526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 266/537] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:31.207860 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d879bd9-8566-4f1b-b1b1-9f5257304600 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:31.249976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:31.257601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8df87-3936-449b-966e-a8b7ac1b1495 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:31.322902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d879bd9-8566-4f1b-b1b1-9f5257304600 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:31.360106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d0cdb-192e-42f8-83ab-1fa8325141ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:31.361093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d0cdb-192e-42f8-83ab-1fa8325141ff None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:31.361383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d0cdb-192e-42f8-83ab-1fa8325141ff None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:31.361663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d0cdb-192e-42f8-83ab-1fa8325141ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:31.369820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d0cdb-192e-42f8-83ab-1fa8325141ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:31.371333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d0cdb-192e-42f8-83ab-1fa8325141ff None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:31.388773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba920a26-4801-4c74-9d2b-ebd779d7732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c24d0cdb-192e-42f8-83ab-1fa8325141ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:31.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:31.389836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:31.390986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:31.390986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:31.390986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:31.390986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:31.390986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:31.390986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:31.390986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:31.390986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:31.390986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 267/538] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:31.392579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920a26-4801-4c74-9d2b-ebd779d7732a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:31.393031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920a26-4801-4c74-9d2b-ebd779d7732a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:31.393449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920a26-4801-4c74-9d2b-ebd779d7732a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:31.394463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920a26-4801-4c74-9d2b-ebd779d7732a None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=316ad1c2eccf4642a21c68856585ceee) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:31.394991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920a26-4801-4c74-9d2b-ebd779d7732a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:31.396284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d0cdb-192e-42f8-83ab-1fa8325141ff None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:31.397084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920a26-4801-4c74-9d2b-ebd779d7732a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:31.417124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 272/539] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:31 2022] GET /identity/v3/role_assignments?user.id=316ad1c2eccf4642a21c68856585ceee&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 259 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:31.506802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:31.514767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d0cdb-192e-42f8-83ab-1fa8325141ff None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:31.565576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e1d8d-2c94-480d-82e7-e347979c8027 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:31.565881 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e1d8d-2c94-480d-82e7-e347979c8027 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:31.566321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e1d8d-2c94-480d-82e7-e347979c8027 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:31.567844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 273/540] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:31.573198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cd45f-ad9e-44b7-b90f-e2822021de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:31.573891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cd45f-ad9e-44b7-b90f-e2822021de53 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:31.574177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cd45f-ad9e-44b7-b90f-e2822021de53 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:31.574481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cd45f-ad9e-44b7-b90f-e2822021de53 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:31.575954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a821f-6ce1-436b-bb73-cf2804e8ec7f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:31.576282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cd45f-ad9e-44b7-b90f-e2822021de53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:31.576607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cd45f-ad9e-44b7-b90f-e2822021de53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:31.576928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a821f-6ce1-436b-bb73-cf2804e8ec7f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:31.577510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a821f-6ce1-436b-bb73-cf2804e8ec7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:31.578096 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cd45f-ad9e-44b7-b90f-e2822021de53 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:31.603442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2f1a821f-6ce1-436b-bb73-cf2804e8ec7f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:31.604929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 268/541] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:31.607241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2f1a821f-6ce1-436b-bb73-cf2804e8ec7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:31.611375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cd45f-ad9e-44b7-b90f-e2822021de53 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:31.726507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 274/542] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:31.757215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a821f-6ce1-436b-bb73-cf2804e8ec7f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:31.758680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cd45f-ad9e-44b7-b90f-e2822021de53 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:31.802916 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:31.834563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08930f-004b-4ff9-b84c-04e21e185b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:31.835225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08930f-004b-4ff9-b84c-04e21e185b0c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:31.835509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08930f-004b-4ff9-b84c-04e21e185b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:31.835885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08930f-004b-4ff9-b84c-04e21e185b0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:31.836725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08930f-004b-4ff9-b84c-04e21e185b0c None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=8b266ea9964c4a179c2743b9cf6478eb) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:31.837042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08930f-004b-4ff9-b84c-04e21e185b0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:31.838502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08930f-004b-4ff9-b84c-04e21e185b0c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:31.852219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 269/543] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:31 2022] GET /identity/v3/role_assignments?user.id=8b266ea9964c4a179c2743b9cf6478eb&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:31.917855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a821f-6ce1-436b-bb73-cf2804e8ec7f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:31.960460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0ee73-cc6d-4cfd-9e5c-f1a0b4970fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:31.961280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0ee73-cc6d-4cfd-9e5c-f1a0b4970fed None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:31.961565 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0ee73-cc6d-4cfd-9e5c-f1a0b4970fed None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:31.961844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0ee73-cc6d-4cfd-9e5c-f1a0b4970fed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:31.974601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0ee73-cc6d-4cfd-9e5c-f1a0b4970fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d9a0ee73-cc6d-4cfd-9e5c-f1a0b4970fed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:31.982785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:31.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:31.984511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 275/544] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:31.991665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08930f-004b-4ff9-b84c-04e21e185b0c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:32.030631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:32.140951 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08930f-004b-4ff9-b84c-04e21e185b0c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:32.180478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1a4be-9755-4bf6-8b9c-4734c788451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:32.181478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1a4be-9755-4bf6-8b9c-4734c788451d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:32.181961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1a4be-9755-4bf6-8b9c-4734c788451d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:32.182462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be1a4be-9755-4bf6-8b9c-4734c788451d None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:32.183539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1a4be-9755-4bf6-8b9c-4734c788451d None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:32.184956 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1a4be-9755-4bf6-8b9c-4734c788451d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:32.186814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be1a4be-9755-4bf6-8b9c-4734c788451d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:32.201808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 270/545] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:32.207478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0ee73-cc6d-4cfd-9e5c-f1a0b4970fed None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:32.343452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:32.349867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0ee73-cc6d-4cfd-9e5c-f1a0b4970fed None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:32.400366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9123ec-0bd2-4c9a-8f8b-4978e416c012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:32.401171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9123ec-0bd2-4c9a-8f8b-4978e416c012 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:32.401456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9123ec-0bd2-4c9a-8f8b-4978e416c012 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:32.401750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9123ec-0bd2-4c9a-8f8b-4978e416c012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:32.413313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9123ec-0bd2-4c9a-8f8b-4978e416c012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:32.414867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9123ec-0bd2-4c9a-8f8b-4978e416c012 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ab9123ec-0bd2-4c9a-8f8b-4978e416c012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:32.436892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:32.437765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:32.439131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:32.439131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:32.439131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:32.439131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:32.439131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 276/546] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:32.444028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1a4be-9755-4bf6-8b9c-4734c788451d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:32.580921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:32.602670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be1a4be-9755-4bf6-8b9c-4734c788451d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:32.652049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc77c6-ff80-4ec3-b92d-de8b2eb4fc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:32.653505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc77c6-ff80-4ec3-b92d-de8b2eb4fc31 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:32.653505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc77c6-ff80-4ec3-b92d-de8b2eb4fc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:32.653793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc77c6-ff80-4ec3-b92d-de8b2eb4fc31 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:32.654332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc77c6-ff80-4ec3-b92d-de8b2eb4fc31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:32.654678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc77c6-ff80-4ec3-b92d-de8b2eb4fc31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:32.656062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc77c6-ff80-4ec3-b92d-de8b2eb4fc31 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:32.677672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 271/547] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:32.682867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9123ec-0bd2-4c9a-8f8b-4978e416c012 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:32.825357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9123ec-0bd2-4c9a-8f8b-4978e416c012 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:32.876193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325ccb8-9232-42d0-a460-bdc9fabc1e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:32.877018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325ccb8-9232-42d0-a460-bdc9fabc1e96 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:32.877301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325ccb8-9232-42d0-a460-bdc9fabc1e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:32.877575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325ccb8-9232-42d0-a460-bdc9fabc1e96 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:32.887821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325ccb8-9232-42d0-a460-bdc9fabc1e96 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:32.889310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325ccb8-9232-42d0-a460-bdc9fabc1e96 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9325ccb8-9232-42d0-a460-bdc9fabc1e96 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:32.914758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:32.915713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:32.916650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 277/548] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:32.922897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc77c6-ff80-4ec3-b92d-de8b2eb4fc31 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:33.062759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc77c6-ff80-4ec3-b92d-de8b2eb4fc31 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:33.111729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442f02a-698d-40e8-ad86-60c26c6ab3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:33.112411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442f02a-698d-40e8-ad86-60c26c6ab3ce None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:33.112696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442f02a-698d-40e8-ad86-60c26c6ab3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:33.112976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442f02a-698d-40e8-ad86-60c26c6ab3ce None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:33.113823 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442f02a-698d-40e8-ad86-60c26c6ab3ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:33.114136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442f02a-698d-40e8-ad86-60c26c6ab3ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:33.115661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442f02a-698d-40e8-ad86-60c26c6ab3ce None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:33.143017 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 272/549] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 12:02:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:33.148253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325ccb8-9232-42d0-a460-bdc9fabc1e96 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:33.159391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:33.275328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325ccb8-9232-42d0-a460-bdc9fabc1e96 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:33.335755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200c1d3-56f7-43e5-b3b4-a04035bc4f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:33.336417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200c1d3-56f7-43e5-b3b4-a04035bc4f9e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:33.336701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200c1d3-56f7-43e5-b3b4-a04035bc4f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:33.336979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200c1d3-56f7-43e5-b3b4-a04035bc4f9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:33.434763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200c1d3-56f7-43e5-b3b4-a04035bc4f9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:33.436354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200c1d3-56f7-43e5-b3b4-a04035bc4f9e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:33.448936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:33.576406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 278/550] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:33.582502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442f02a-698d-40e8-ad86-60c26c6ab3ce None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:33.585720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:33.593676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf958-e7f7-4237-ba77-25052a8d1fc4 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:33.595973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf958-e7f7-4237-ba77-25052a8d1fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:33.596480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf958-e7f7-4237-ba77-25052a8d1fc4 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:33.600411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 279/551] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:33 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:33.606374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88524a92-c448-414f-972b-777785566505 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:33.607002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88524a92-c448-414f-972b-777785566505 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:33.607479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88524a92-c448-414f-972b-777785566505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:33.637698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-88524a92-c448-414f-972b-777785566505 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:33.640379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-88524a92-c448-414f-972b-777785566505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:33.759848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442f02a-698d-40e8-ad86-60c26c6ab3ce None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:33.772389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 280/552] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:33.804778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88524a92-c448-414f-972b-777785566505 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:33.848643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5accd32-7a9a-4901-a29d-41426f91d631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:33.849452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5accd32-7a9a-4901-a29d-41426f91d631 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:33.849737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5accd32-7a9a-4901-a29d-41426f91d631 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:33.850013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5accd32-7a9a-4901-a29d-41426f91d631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:33.861343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5accd32-7a9a-4901-a29d-41426f91d631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:33.862872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5accd32-7a9a-4901-a29d-41426f91d631 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:33.868458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-e5accd32-7a9a-4901-a29d-41426f91d631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:33.881358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:33.886542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:33.888532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:33.888532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:33.888532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:33.888532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:33.888532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 273/553] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:33.889257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5accd32-7a9a-4901-a29d-41426f91d631 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:33.988784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88524a92-c448-414f-972b-777785566505 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:33.999655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:34.052867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc4e8b-366b-429e-9540-f1f12490ad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.054354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4e8b-366b-429e-9540-f1f12490ad76 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.054676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4e8b-366b-429e-9540-f1f12490ad76 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.054959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4e8b-366b-429e-9540-f1f12490ad76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.061625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5accd32-7a9a-4901-a29d-41426f91d631 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:34.069411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc4e8b-366b-429e-9540-f1f12490ad76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f1bc4e8b-366b-429e-9540-f1f12490ad76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:34.079800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:34.080775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:34.081654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 281/554] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:34.082895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c37dc6-8d0b-4b6b-ba1d-f95be305ecc6 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.082993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c37dc6-8d0b-4b6b-ba1d-f95be305ecc6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.083258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c37dc6-8d0b-4b6b-ba1d-f95be305ecc6 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.088929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 282/555] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:34.088929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c37dc6-8d0b-4b6b-ba1d-f95be305ecc6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:34.111061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b08392-8cbe-4c35-8885-d193e3ee5376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.112170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b08392-8cbe-4c35-8885-d193e3ee5376 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.112738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b08392-8cbe-4c35-8885-d193e3ee5376 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.113237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b08392-8cbe-4c35-8885-d193e3ee5376 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.114156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b08392-8cbe-4c35-8885-d193e3ee5376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:34.114686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b08392-8cbe-4c35-8885-d193e3ee5376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:34.116354 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b08392-8cbe-4c35-8885-d193e3ee5376 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:34.130323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 274/556] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:34.132204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5efa53-5eeb-40bd-8c4e-ece5bc2c75a0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.132751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5efa53-5eeb-40bd-8c4e-ece5bc2c75a0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.133228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5efa53-5eeb-40bd-8c4e-ece5bc2c75a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.152393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0f5efa53-5eeb-40bd-8c4e-ece5bc2c75a0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:34.155260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0f5efa53-5eeb-40bd-8c4e-ece5bc2c75a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:34.208001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:34.213265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c37dc6-8d0b-4b6b-ba1d-f95be305ecc6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:34.233338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 275/557] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:34.234585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5efa53-5eeb-40bd-8c4e-ece5bc2c75a0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:34.260164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9a07-6b4e-4fad-9709-47e16c66ea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.261302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9a07-6b4e-4fad-9709-47e16c66ea94 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.261793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9a07-6b4e-4fad-9709-47e16c66ea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.262494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9a07-6b4e-4fad-9709-47e16c66ea94 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.263578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9a07-6b4e-4fad-9709-47e16c66ea94 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:34.264037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9a07-6b4e-4fad-9709-47e16c66ea94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:34.265564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9a07-6b4e-4fad-9709-47e16c66ea94 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:34.274495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 283/558] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:34.276017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c4f2e-bae4-4c0e-a66c-ae57439384ae None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.276893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c4f2e-bae4-4c0e-a66c-ae57439384ae None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.277313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c4f2e-bae4-4c0e-a66c-ae57439384ae None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.279925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 284/559] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:34.280790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c4f2e-bae4-4c0e-a66c-ae57439384ae None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:34.294000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:34.350346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5efa53-5eeb-40bd-8c4e-ece5bc2c75a0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:34.390206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab43c0-aecc-4641-a6c0-9f1a7bcd52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.391037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab43c0-aecc-4641-a6c0-9f1a7bcd52ff None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.391321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab43c0-aecc-4641-a6c0-9f1a7bcd52ff None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.391801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab43c0-aecc-4641-a6c0-9f1a7bcd52ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.398533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab43c0-aecc-4641-a6c0-9f1a7bcd52ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:34.399034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c4f2e-bae4-4c0e-a66c-ae57439384ae None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:34.399979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab43c0-aecc-4641-a6c0-9f1a7bcd52ff None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b0ab43c0-aecc-4641-a6c0-9f1a7bcd52ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:34.414672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:34.415584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:34.416441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:34.416441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:34.416441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:34.416441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:34.416441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:34.416441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:34.416441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:34.416441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:34.416441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 276/560] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:34.417177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab43c0-aecc-4641-a6c0-9f1a7bcd52ff None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:34.444210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd60a76-c5c5-4234-a838-1536b69e5200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.445010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd60a76-c5c5-4234-a838-1536b69e5200 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.445293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd60a76-c5c5-4234-a838-1536b69e5200 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.445573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd60a76-c5c5-4234-a838-1536b69e5200 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.457483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd60a76-c5c5-4234-a838-1536b69e5200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9fd60a76-c5c5-4234-a838-1536b69e5200 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:34.465462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:34.466485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:34.467299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 285/561] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:34.468445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780882b-e970-4928-9223-27676d415090 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.468541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780882b-e970-4928-9223-27676d415090 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.468804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780882b-e970-4928-9223-27676d415090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.487070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9780882b-e970-4928-9223-27676d415090 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:34.488234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9780882b-e970-4928-9223-27676d415090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:34.503196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:34.537927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab43c0-aecc-4641-a6c0-9f1a7bcd52ff None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:34.540622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:34.560061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 286/562] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:34.561557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780882b-e970-4928-9223-27676d415090 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:34.576510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2103a-d011-4820-aba6-9ad503ff984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.577306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2103a-d011-4820-aba6-9ad503ff984f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.577613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2103a-d011-4820-aba6-9ad503ff984f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.577871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2103a-d011-4820-aba6-9ad503ff984f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.589844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2103a-d011-4820-aba6-9ad503ff984f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:34.591443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc2103a-d011-4820-aba6-9ad503ff984f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0cc2103a-d011-4820-aba6-9ad503ff984f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:34.609370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:34.610476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:34.611740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:34.611740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:34.611740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:34.611740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:34.614606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 277/563] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:34.615674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2103a-d011-4820-aba6-9ad503ff984f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:34.673143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780882b-e970-4928-9223-27676d415090 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:34.706920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f78b6-0851-425a-9204-262b1ca205e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.707594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f78b6-0851-425a-9204-262b1ca205e0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.707879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f78b6-0851-425a-9204-262b1ca205e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.708161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f78b6-0851-425a-9204-262b1ca205e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.708942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f78b6-0851-425a-9204-262b1ca205e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:34.709283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f78b6-0851-425a-9204-262b1ca205e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:34.710733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f78b6-0851-425a-9204-262b1ca205e0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:34.715448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2103a-d011-4820-aba6-9ad503ff984f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:34.722529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 287/564] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:34.723644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf65cde-9e00-4729-a9d6-98cd1142d019 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.723938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf65cde-9e00-4729-a9d6-98cd1142d019 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.724221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf65cde-9e00-4729-a9d6-98cd1142d019 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.725583 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 288/565] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:34.726164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf65cde-9e00-4729-a9d6-98cd1142d019 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:34.740457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:34.752418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ff53e-9c40-48d6-8f5e-23dd3386871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.753081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ff53e-9c40-48d6-8f5e-23dd3386871e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.753366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ff53e-9c40-48d6-8f5e-23dd3386871e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.753646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ff53e-9c40-48d6-8f5e-23dd3386871e None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.760179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ff53e-9c40-48d6-8f5e-23dd3386871e None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:34.760179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ff53e-9c40-48d6-8f5e-23dd3386871e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:34.760179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ff53e-9c40-48d6-8f5e-23dd3386871e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:34.764691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 278/566] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:34.765402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ff53e-9c40-48d6-8f5e-23dd3386871e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:34.783963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:34.833411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf65cde-9e00-4729-a9d6-98cd1142d019 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:34.870097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ff53e-9c40-48d6-8f5e-23dd3386871e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:34.873686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0af3d-5edc-4369-ac56-bac6ea211c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.874522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0af3d-5edc-4369-ac56-bac6ea211c25 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.874807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0af3d-5edc-4369-ac56-bac6ea211c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.875082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0af3d-5edc-4369-ac56-bac6ea211c25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.885428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0af3d-5edc-4369-ac56-bac6ea211c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-4eb0af3d-5edc-4369-ac56-bac6ea211c25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:34.893910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:34.894996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:34.895828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 289/567] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:34.896536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0af3d-5edc-4369-ac56-bac6ea211c25 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:34.909530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4f41-fca4-4c59-9ea6-84b30b4ab09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:34.910246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4f41-fca4-4c59-9ea6-84b30b4ab09e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.910567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4f41-fca4-4c59-9ea6-84b30b4ab09e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.910864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4f41-fca4-4c59-9ea6-84b30b4ab09e None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.911579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d4f41-fca4-4c59-9ea6-84b30b4ab09e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:34.911896 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d4f41-fca4-4c59-9ea6-84b30b4ab09e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:34.913259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d4f41-fca4-4c59-9ea6-84b30b4ab09e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:34.925810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 279/568] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:34.927085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d7d28-9103-4d2a-a73c-e3f15fff7725 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:34.927384 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d7d28-9103-4d2a-a73c-e3f15fff7725 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:34.927667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d7d28-9103-4d2a-a73c-e3f15fff7725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:34.944538 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6e6d7d28-9103-4d2a-a73c-e3f15fff7725 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:34.945801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6e6d7d28-9103-4d2a-a73c-e3f15fff7725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:34.972666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:35.005314 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0af3d-5edc-4369-ac56-bac6ea211c25 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.026431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 280/569] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:35.026431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d7d28-9103-4d2a-a73c-e3f15fff7725 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.026431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:35.047183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5556b-f742-4598-8a96-0128eb56f22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.047845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5556b-f742-4598-8a96-0128eb56f22e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.048135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5556b-f742-4598-8a96-0128eb56f22e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.048413 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5556b-f742-4598-8a96-0128eb56f22e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.049249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5556b-f742-4598-8a96-0128eb56f22e None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=28ddf898ad7b4e10aaf0770249b4453d) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:35.049563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5556b-f742-4598-8a96-0128eb56f22e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.051046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a5556b-f742-4598-8a96-0128eb56f22e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.056669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 290/570] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:34 2022] GET /identity/v3/role_assignments?user.id=28ddf898ad7b4e10aaf0770249b4453d&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:35.057466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5556b-f742-4598-8a96-0128eb56f22e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.129827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d7d28-9103-4d2a-a73c-e3f15fff7725 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.171004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50d9b1-8e63-48f1-9e45-6efb607f32cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.171901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50d9b1-8e63-48f1-9e45-6efb607f32cc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.172185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50d9b1-8e63-48f1-9e45-6efb607f32cc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.172461 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50d9b1-8e63-48f1-9e45-6efb607f32cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.174909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5556b-f742-4598-8a96-0128eb56f22e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.189374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50d9b1-8e63-48f1-9e45-6efb607f32cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.191004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50d9b1-8e63-48f1-9e45-6efb607f32cc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-aa50d9b1-8e63-48f1-9e45-6efb607f32cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:35.212144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:35.213661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:35.215875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:35.215875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:35.215875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:35.215875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:35.215875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 281/571] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:35.215875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50d9b1-8e63-48f1-9e45-6efb607f32cc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.225444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a611fed-e62d-4229-a8ad-27a3119e6aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.226133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a611fed-e62d-4229-a8ad-27a3119e6aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.226446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a611fed-e62d-4229-a8ad-27a3119e6aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.226721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a611fed-e62d-4229-a8ad-27a3119e6aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.227492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a611fed-e62d-4229-a8ad-27a3119e6aa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:35.227790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a611fed-e62d-4229-a8ad-27a3119e6aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.229186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a611fed-e62d-4229-a8ad-27a3119e6aa3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.231493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:35.240229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 291/572] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:35.247276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a611fed-e62d-4229-a8ad-27a3119e6aa3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.288359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:35.330431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50d9b1-8e63-48f1-9e45-6efb607f32cc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.352415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a611fed-e62d-4229-a8ad-27a3119e6aa3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.366092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88b879-5fc0-46f3-a512-33b6bca1c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.366918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88b879-5fc0-46f3-a512-33b6bca1c219 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.367200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88b879-5fc0-46f3-a512-33b6bca1c219 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.367470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88b879-5fc0-46f3-a512-33b6bca1c219 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.375786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88b879-5fc0-46f3-a512-33b6bca1c219 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.377215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d88b879-5fc0-46f3-a512-33b6bca1c219 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4d88b879-5fc0-46f3-a512-33b6bca1c219 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:35.392527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:35.393492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:35.394453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 282/573] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:35.395134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88b879-5fc0-46f3-a512-33b6bca1c219 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.395752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada8ac7-3fda-4a10-94c3-c19919e2a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.396674 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8ac7-3fda-4a10-94c3-c19919e2a169 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.397053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8ac7-3fda-4a10-94c3-c19919e2a169 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.397331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8ac7-3fda-4a10-94c3-c19919e2a169 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.410613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada8ac7-3fda-4a10-94c3-c19919e2a169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-fada8ac7-3fda-4a10-94c3-c19919e2a169 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:35.419923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:35.420904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:35.421728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 292/574] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:35.426494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada8ac7-3fda-4a10-94c3-c19919e2a169 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.486465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:35.515293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88b879-5fc0-46f3-a512-33b6bca1c219 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.533716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada8ac7-3fda-4a10-94c3-c19919e2a169 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.537512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:35.552685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f9611-fb1f-4524-9a4d-2f2bec7b05b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.553353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f9611-fb1f-4524-9a4d-2f2bec7b05b5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.553635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f9611-fb1f-4524-9a4d-2f2bec7b05b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.553908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f9611-fb1f-4524-9a4d-2f2bec7b05b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.554815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f9611-fb1f-4524-9a4d-2f2bec7b05b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:35.555124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f9611-fb1f-4524-9a4d-2f2bec7b05b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.556484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f9611-fb1f-4524-9a4d-2f2bec7b05b5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.570779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 283/575] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:35.572133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f9611-fb1f-4524-9a4d-2f2bec7b05b5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.593104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e80b-e6a9-4018-ba19-f0c29753f1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.593909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e80b-e6a9-4018-ba19-f0c29753f1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.594195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e80b-e6a9-4018-ba19-f0c29753f1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.594510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e80b-e6a9-4018-ba19-f0c29753f1d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.605887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397e80b-e6a9-4018-ba19-f0c29753f1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.607400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397e80b-e6a9-4018-ba19-f0c29753f1d1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-2397e80b-e6a9-4018-ba19-f0c29753f1d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:35.625626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:35.626598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:35.627454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:35.627454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:35.627454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:35.627454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:35.627454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 293/576] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:35.628190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e80b-e6a9-4018-ba19-f0c29753f1d1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.688011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f9611-fb1f-4524-9a4d-2f2bec7b05b5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.724628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255142d5-ad73-4f30-b883-d982fc556d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.725789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255142d5-ad73-4f30-b883-d982fc556d84 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.726339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255142d5-ad73-4f30-b883-d982fc556d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.726870 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255142d5-ad73-4f30-b883-d982fc556d84 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.727837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255142d5-ad73-4f30-b883-d982fc556d84 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:35.728246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255142d5-ad73-4f30-b883-d982fc556d84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.729824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255142d5-ad73-4f30-b883-d982fc556d84 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.732224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e80b-e6a9-4018-ba19-f0c29753f1d1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.741231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:35.749774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 284/577] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:35.750756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255142d5-ad73-4f30-b883-d982fc556d84 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.772227 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a501ccc-e6b5-4804-a50e-f8c1ff210fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.773227 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a501ccc-e6b5-4804-a50e-f8c1ff210fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.773697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a501ccc-e6b5-4804-a50e-f8c1ff210fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.774175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a501ccc-e6b5-4804-a50e-f8c1ff210fd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.774967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a501ccc-e6b5-4804-a50e-f8c1ff210fd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:35.775284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a501ccc-e6b5-4804-a50e-f8c1ff210fd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.776655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a501ccc-e6b5-4804-a50e-f8c1ff210fd5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.785136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 294/578] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:35.785935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a501ccc-e6b5-4804-a50e-f8c1ff210fd5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.798393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:35.860416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255142d5-ad73-4f30-b883-d982fc556d84 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.894118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a501ccc-e6b5-4804-a50e-f8c1ff210fd5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:35.895703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4cbbf-237c-4f29-ace8-6ff765c675c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.896506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4cbbf-237c-4f29-ace8-6ff765c675c7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.896792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4cbbf-237c-4f29-ace8-6ff765c675c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.897060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4cbbf-237c-4f29-ace8-6ff765c675c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.905090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4cbbf-237c-4f29-ace8-6ff765c675c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.906576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4cbbf-237c-4f29-ace8-6ff765c675c7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0ff4cbbf-237c-4f29-ace8-6ff765c675c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:35.921989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:35.923218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 285/579] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:35.924156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4cbbf-237c-4f29-ace8-6ff765c675c7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.937025 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e25ac3-d9ec-4521-8338-0d0fde4c86fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:35.937693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e25ac3-d9ec-4521-8338-0d0fde4c86fb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:35.937982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e25ac3-d9ec-4521-8338-0d0fde4c86fb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:35.938314 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e25ac3-d9ec-4521-8338-0d0fde4c86fb None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:35.939023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e25ac3-d9ec-4521-8338-0d0fde4c86fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:35.939333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e25ac3-d9ec-4521-8338-0d0fde4c86fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:35.940701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e25ac3-d9ec-4521-8338-0d0fde4c86fb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:35.952836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 295/580] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:35.953505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e25ac3-d9ec-4521-8338-0d0fde4c86fb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:35.984714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:36.032302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:36.036236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4cbbf-237c-4f29-ace8-6ff765c675c7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.069633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e25ac3-d9ec-4521-8338-0d0fde4c86fb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.073563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674643f-ebb9-4ec6-b8d6-e1a90c2ed698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.075214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674643f-ebb9-4ec6-b8d6-e1a90c2ed698 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.075644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674643f-ebb9-4ec6-b8d6-e1a90c2ed698 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.076059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674643f-ebb9-4ec6-b8d6-e1a90c2ed698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.090914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674643f-ebb9-4ec6-b8d6-e1a90c2ed698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.092842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674643f-ebb9-4ec6-b8d6-e1a90c2ed698 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.106114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650726e-3ea0-4ecf-a99d-ff7e90ed5f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.107046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650726e-3ea0-4ecf-a99d-ff7e90ed5f6e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.107556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650726e-3ea0-4ecf-a99d-ff7e90ed5f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.107950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650726e-3ea0-4ecf-a99d-ff7e90ed5f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1674643f-ebb9-4ec6-b8d6-e1a90c2ed698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:36.112053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:36.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:36.119015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:36.119015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:36.119015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:36.119015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:36.119015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 286/581] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:36.119015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674643f-ebb9-4ec6-b8d6-e1a90c2ed698 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:36.123106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650726e-3ea0-4ecf-a99d-ff7e90ed5f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.124801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650726e-3ea0-4ecf-a99d-ff7e90ed5f6e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-0650726e-3ea0-4ecf-a99d-ff7e90ed5f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:36.144657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:36.145561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:36.146461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:36.146461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:36.146461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:36.146461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:36.146461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 296/582] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:36.146748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650726e-3ea0-4ecf-a99d-ff7e90ed5f6e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:36.237774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:36.243810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674643f-ebb9-4ec6-b8d6-e1a90c2ed698 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.256750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650726e-3ea0-4ecf-a99d-ff7e90ed5f6e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.283149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fccd55-7e38-4d12-b1b0-986efcdfaf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.283841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fccd55-7e38-4d12-b1b0-986efcdfaf76 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.284142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fccd55-7e38-4d12-b1b0-986efcdfaf76 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.284435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fccd55-7e38-4d12-b1b0-986efcdfaf76 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.285332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fccd55-7e38-4d12-b1b0-986efcdfaf76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:36.285644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fccd55-7e38-4d12-b1b0-986efcdfaf76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.287157 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fccd55-7e38-4d12-b1b0-986efcdfaf76 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.293877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a41b99-edd5-4041-947e-046b7d982dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.294950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41b99-edd5-4041-947e-046b7d982dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.295414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41b99-edd5-4041-947e-046b7d982dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.295866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a41b99-edd5-4041-947e-046b7d982dc8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.304543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a41b99-edd5-4041-947e-046b7d982dc8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.306222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a41b99-edd5-4041-947e-046b7d982dc8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.306827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 287/583] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 12:02:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:36.307674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fccd55-7e38-4d12-b1b0-986efcdfaf76 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:36.321750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-54a41b99-edd5-4041-947e-046b7d982dc8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:36.326785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:36.327777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:36.328728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 297/584] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:36.329344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a41b99-edd5-4041-947e-046b7d982dc8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:36.428224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fccd55-7e38-4d12-b1b0-986efcdfaf76 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.467112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbd018-a258-4bab-9e23-411f2197f07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.467892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbd018-a258-4bab-9e23-411f2197f07f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.468177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbd018-a258-4bab-9e23-411f2197f07f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.468458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fbd018-a258-4bab-9e23-411f2197f07f None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.469185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbd018-a258-4bab-9e23-411f2197f07f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:36.469497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbd018-a258-4bab-9e23-411f2197f07f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.470892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fbd018-a258-4bab-9e23-411f2197f07f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.488375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 288/585] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:36.488375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbd018-a258-4bab-9e23-411f2197f07f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:36.561557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:36.565618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a41b99-edd5-4041-947e-046b7d982dc8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.598552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fbd018-a258-4bab-9e23-411f2197f07f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.607358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50e7a7-8ff5-4b02-89d8-c2a5990e5ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.608030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50e7a7-8ff5-4b02-89d8-c2a5990e5ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.608317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50e7a7-8ff5-4b02-89d8-c2a5990e5ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.615344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50e7a7-8ff5-4b02-89d8-c2a5990e5ce2 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.615344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50e7a7-8ff5-4b02-89d8-c2a5990e5ce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:36.615344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50e7a7-8ff5-4b02-89d8-c2a5990e5ce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.615344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50e7a7-8ff5-4b02-89d8-c2a5990e5ce2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.628900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 298/586] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:36.628900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50e7a7-8ff5-4b02-89d8-c2a5990e5ce2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:36.628900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:36.643897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adc3a4-a0aa-439d-83be-c2466e6fe1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.644874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc3a4-a0aa-439d-83be-c2466e6fe1ef None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.645299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc3a4-a0aa-439d-83be-c2466e6fe1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.645656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc3a4-a0aa-439d-83be-c2466e6fe1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.658845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adc3a4-a0aa-439d-83be-c2466e6fe1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.660476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adc3a4-a0aa-439d-83be-c2466e6fe1ef None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-18adc3a4-a0aa-439d-83be-c2466e6fe1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:36.678627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:36.679712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:36.680707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:36.680707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:36.680707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:36.680707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:36.680707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 289/587] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:36.685098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adc3a4-a0aa-439d-83be-c2466e6fe1ef None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:36.739196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50e7a7-8ff5-4b02-89d8-c2a5990e5ce2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.775845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38080f51-3221-4199-a6eb-b36b12201248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.776766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38080f51-3221-4199-a6eb-b36b12201248 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.777194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38080f51-3221-4199-a6eb-b36b12201248 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.778213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38080f51-3221-4199-a6eb-b36b12201248 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.779345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38080f51-3221-4199-a6eb-b36b12201248 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:36.779808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38080f51-3221-4199-a6eb-b36b12201248 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.781386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38080f51-3221-4199-a6eb-b36b12201248 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.795308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adc3a4-a0aa-439d-83be-c2466e6fe1ef None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.801035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 299/588] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 12:02:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:36.802178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38080f51-3221-4199-a6eb-b36b12201248 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:36.819756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:36.834897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24daa-407d-4a2a-89c0-b0898f123dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.835771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24daa-407d-4a2a-89c0-b0898f123dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.836156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24daa-407d-4a2a-89c0-b0898f123dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.836482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24daa-407d-4a2a-89c0-b0898f123dd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.844241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c24daa-407d-4a2a-89c0-b0898f123dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.845743 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c24daa-407d-4a2a-89c0-b0898f123dd6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-41c24daa-407d-4a2a-89c0-b0898f123dd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:36.860761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:36.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:36.863085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:36.863085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:36.863085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:36.863085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:36.863085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:36.863085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:36.863085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:36.863085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:36.863085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 290/589] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:36.867385 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24daa-407d-4a2a-89c0-b0898f123dd6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:36.875010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:36.930226 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38080f51-3221-4199-a6eb-b36b12201248 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.973169 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0789e6f-b7fd-4bca-a166-155500a92290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:36.974023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0789e6f-b7fd-4bca-a166-155500a92290 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:36.974306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0789e6f-b7fd-4bca-a166-155500a92290 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:36.974612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0789e6f-b7fd-4bca-a166-155500a92290 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:36.982658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0789e6f-b7fd-4bca-a166-155500a92290 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:36.984104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0789e6f-b7fd-4bca-a166-155500a92290 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:36.993591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24daa-407d-4a2a-89c0-b0898f123dd6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a0789e6f-b7fd-4bca-a166-155500a92290 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:36.999430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:37.000337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 300/590] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:37.006247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0789e6f-b7fd-4bca-a166-155500a92290 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.048363 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0db1be-d242-4a79-8a98-e3d95ebb92d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.049025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0db1be-d242-4a79-8a98-e3d95ebb92d1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.049306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0db1be-d242-4a79-8a98-e3d95ebb92d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.049581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0db1be-d242-4a79-8a98-e3d95ebb92d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.050302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0db1be-d242-4a79-8a98-e3d95ebb92d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:37.050659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0db1be-d242-4a79-8a98-e3d95ebb92d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.052023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0db1be-d242-4a79-8a98-e3d95ebb92d1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.069381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 291/591] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:37.069381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0db1be-d242-4a79-8a98-e3d95ebb92d1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.114734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:37.130136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0789e6f-b7fd-4bca-a166-155500a92290 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:37.155805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:37.174011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76bc7e-5188-44a0-bd15-d14c0a6684ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.174983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76bc7e-5188-44a0-bd15-d14c0a6684ab None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.175269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76bc7e-5188-44a0-bd15-d14c0a6684ab None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.175545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76bc7e-5188-44a0-bd15-d14c0a6684ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.187036 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76bc7e-5188-44a0-bd15-d14c0a6684ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.187959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0db1be-d242-4a79-8a98-e3d95ebb92d1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:37.188545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76bc7e-5188-44a0-bd15-d14c0a6684ab None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ef76bc7e-5188-44a0-bd15-d14c0a6684ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:37.206908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:37.207836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:37.208689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:37.208689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:37.208689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:37.208689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:37.212839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 301/592] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:37.213699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76bc7e-5188-44a0-bd15-d14c0a6684ab None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.225382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a87f1-4ab4-4faf-9796-6a2eb1963b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.226051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a87f1-4ab4-4faf-9796-6a2eb1963b6f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.226338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a87f1-4ab4-4faf-9796-6a2eb1963b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.226670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a87f1-4ab4-4faf-9796-6a2eb1963b6f None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.227470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a87f1-4ab4-4faf-9796-6a2eb1963b6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:37.227796 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a87f1-4ab4-4faf-9796-6a2eb1963b6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.229693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a87f1-4ab4-4faf-9796-6a2eb1963b6f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.247532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 292/593] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:37.248400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a87f1-4ab4-4faf-9796-6a2eb1963b6f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.345838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76bc7e-5188-44a0-bd15-d14c0a6684ab None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:37.371356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:37.374262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a87f1-4ab4-4faf-9796-6a2eb1963b6f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:37.380537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c9cfd-e603-4f3b-a6c2-bd82218331b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.381215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c9cfd-e603-4f3b-a6c2-bd82218331b0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.381485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c9cfd-e603-4f3b-a6c2-bd82218331b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.381760 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c9cfd-e603-4f3b-a6c2-bd82218331b0 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.382651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c9cfd-e603-4f3b-a6c2-bd82218331b0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:37.388224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c9cfd-e603-4f3b-a6c2-bd82218331b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.388224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c9cfd-e603-4f3b-a6c2-bd82218331b0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.401530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 302/594] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 12:02:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:37.403072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c9cfd-e603-4f3b-a6c2-bd82218331b0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.412306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbf11e-dd34-4c01-9834-b75c963dd613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.413119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbf11e-dd34-4c01-9834-b75c963dd613 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.413403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbf11e-dd34-4c01-9834-b75c963dd613 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.413686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbf11e-dd34-4c01-9834-b75c963dd613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.421790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbf11e-dd34-4c01-9834-b75c963dd613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.423665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbf11e-dd34-4c01-9834-b75c963dd613 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.433419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6dcbf11e-dd34-4c01-9834-b75c963dd613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:37.437925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:37.438970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:37.439884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:37.439884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:37.439884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:37.439884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:37.439884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:37.439884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:37.439884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:37.439884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:37.439884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 293/595] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:37.440358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbf11e-dd34-4c01-9834-b75c963dd613 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.507137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c9cfd-e603-4f3b-a6c2-bd82218331b0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:37.540746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3281e-aa5c-451d-8a82-8386074ec3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.541410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3281e-aa5c-451d-8a82-8386074ec3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.541696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3281e-aa5c-451d-8a82-8386074ec3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.542053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbf11e-dd34-4c01-9834-b75c963dd613 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:37.542238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3281e-aa5c-451d-8a82-8386074ec3e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.543501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3281e-aa5c-451d-8a82-8386074ec3e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:37.544710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3281e-aa5c-451d-8a82-8386074ec3e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.545541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3281e-aa5c-451d-8a82-8386074ec3e8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.557210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 303/596] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:37.558546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b0a02-7dfe-4808-b2e3-bdb271e16848 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.559576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b0a02-7dfe-4808-b2e3-bdb271e16848 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.559889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b0a02-7dfe-4808-b2e3-bdb271e16848 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.561347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 304/597] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:37.562124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b0a02-7dfe-4808-b2e3-bdb271e16848 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.586355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa768c3d-60af-43a3-af56-dae046022e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.587297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa768c3d-60af-43a3-af56-dae046022e75 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.588606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa768c3d-60af-43a3-af56-dae046022e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.588606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa768c3d-60af-43a3-af56-dae046022e75 None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/9969632e796345fa9671a45cf61ede3e/roles/941f412208c04a7abd395240a0339304` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.605232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa768c3d-60af-43a3-af56-dae046022e75 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=9969632e796345fa9671a45cf61ede3e, role_id=941f412208c04a7abd395240a0339304)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.607859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa768c3d-60af-43a3-af56-dae046022e75 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.654048 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:37.664087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b0a02-7dfe-4808-b2e3-bdb271e16848 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:37.697268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac8395-fc55-4989-860a-efebeb14d50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.698056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac8395-fc55-4989-860a-efebeb14d50c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.698341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac8395-fc55-4989-860a-efebeb14d50c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.698645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac8395-fc55-4989-860a-efebeb14d50c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.700525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 294/598] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:02:37 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/9969632e796345fa9671a45cf61ede3e/roles/941f412208c04a7abd395240a0339304 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:02:37.712885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa768c3d-60af-43a3-af56-dae046022e75 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.713040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac8395-fc55-4989-860a-efebeb14d50c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.713040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac8395-fc55-4989-860a-efebeb14d50c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.732291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-18ac8395-fc55-4989-860a-efebeb14d50c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:37.732911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:37.733854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:37.739106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:37.739106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:37.739106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:37.739106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:37.739106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 305/599] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:37.739106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac8395-fc55-4989-860a-efebeb14d50c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.843820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa768c3d-60af-43a3-af56-dae046022e75 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:37.851595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac8395-fc55-4989-860a-efebeb14d50c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:37.886313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c59623-554c-41b2-96e1-1b9f8a34429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.886313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c59623-554c-41b2-96e1-1b9f8a34429b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.886313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c59623-554c-41b2-96e1-1b9f8a34429b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.886694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c59623-554c-41b2-96e1-1b9f8a34429b None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.887942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c59623-554c-41b2-96e1-1b9f8a34429b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:37.889026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c59623-554c-41b2-96e1-1b9f8a34429b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.890648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c59623-554c-41b2-96e1-1b9f8a34429b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.894988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207509ae-a4b4-49b3-8fbf-11777499d883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:37.895796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207509ae-a4b4-49b3-8fbf-11777499d883 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.896076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207509ae-a4b4-49b3-8fbf-11777499d883 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.896349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207509ae-a4b4-49b3-8fbf-11777499d883 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.902689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 295/600] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:37.903974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5cd6d-e73a-4ca0-aa15-8425380f7c01 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:37.904267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5cd6d-e73a-4ca0-aa15-8425380f7c01 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:37.904552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5cd6d-e73a-4ca0-aa15-8425380f7c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:37.905441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207509ae-a4b4-49b3-8fbf-11777499d883 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:37.907162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207509ae-a4b4-49b3-8fbf-11777499d883 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:37.922000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0ac5cd6d-e73a-4ca0-aa15-8425380f7c01 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:37.923241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0ac5cd6d-e73a-4ca0-aa15-8425380f7c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-207509ae-a4b4-49b3-8fbf-11777499d883 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:37.923976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:37.924845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:37.925736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:37.925736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:37.925736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:37.925736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:37.925736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:37.925736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:37.925736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:37.925736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:37.925736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 306/601] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:37.930582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207509ae-a4b4-49b3-8fbf-11777499d883 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:37.934328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:37.992037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:37.997569 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 296/602] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:37.998380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5cd6d-e73a-4ca0-aa15-8425380f7c01 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:38.054190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207509ae-a4b4-49b3-8fbf-11777499d883 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:38.089443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68096500-b7f3-4efd-bc05-6e50b9699073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:38.090262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68096500-b7f3-4efd-bc05-6e50b9699073 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:38.090646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68096500-b7f3-4efd-bc05-6e50b9699073 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:38.090931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68096500-b7f3-4efd-bc05-6e50b9699073 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:38.091701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68096500-b7f3-4efd-bc05-6e50b9699073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:38.092034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68096500-b7f3-4efd-bc05-6e50b9699073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:38.093415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68096500-b7f3-4efd-bc05-6e50b9699073 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:38.105481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 307/603] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:38.106294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68096500-b7f3-4efd-bc05-6e50b9699073 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:38.133787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5cd6d-e73a-4ca0-aa15-8425380f7c01 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:38.172108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d3b03-1609-4458-994a-0d5e72e4a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:38.173306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d3b03-1609-4458-994a-0d5e72e4a8cc None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:38.173784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d3b03-1609-4458-994a-0d5e72e4a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:38.174066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d3b03-1609-4458-994a-0d5e72e4a8cc None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/bb6776311f354b748d563c4f1d793972/roles/941f412208c04a7abd395240a0339304` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:38.176953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:38.195188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d3b03-1609-4458-994a-0d5e72e4a8cc None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=bb6776311f354b748d563c4f1d793972, role_id=941f412208c04a7abd395240a0339304)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:38.196661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d3b03-1609-4458-994a-0d5e72e4a8cc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:38.218924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68096500-b7f3-4efd-bc05-6e50b9699073 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:38.254263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2358d37-e2fc-40f9-afd6-abce5b2e93a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:38.254968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2358d37-e2fc-40f9-afd6-abce5b2e93a8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:38.255255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2358d37-e2fc-40f9-afd6-abce5b2e93a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:38.255534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2358d37-e2fc-40f9-afd6-abce5b2e93a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:38.256324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2358d37-e2fc-40f9-afd6-abce5b2e93a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:38.256641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2358d37-e2fc-40f9-afd6-abce5b2e93a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:38.258078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2358d37-e2fc-40f9-afd6-abce5b2e93a8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:38.269878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 308/604] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:38.270786 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2358d37-e2fc-40f9-afd6-abce5b2e93a8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:38.289080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 297/605] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:02:37 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/bb6776311f354b748d563c4f1d793972/roles/941f412208c04a7abd395240a0339304 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:02:38.290617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d3b03-1609-4458-994a-0d5e72e4a8cc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:38.361846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:38.417276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2358d37-e2fc-40f9-afd6-abce5b2e93a8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:38.442223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d3b03-1609-4458-994a-0d5e72e4a8cc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:38.454909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:38.463808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a927b36-aff5-46d7-815b-7f994aa0f348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:38.465692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927b36-aff5-46d7-815b-7f994aa0f348 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:38.466180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927b36-aff5-46d7-815b-7f994aa0f348 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:38.466769 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927b36-aff5-46d7-815b-7f994aa0f348 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:38.482371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a927b36-aff5-46d7-815b-7f994aa0f348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:38.487853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df80a39-236f-4db0-a100-6264bdb82f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:38.488907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df80a39-236f-4db0-a100-6264bdb82f3b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:38.489404 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df80a39-236f-4db0-a100-6264bdb82f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:38.489875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df80a39-236f-4db0-a100-6264bdb82f3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-4a927b36-aff5-46d7-815b-7f994aa0f348 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:38.490968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:38.491877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:38.492713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 309/606] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:38.495133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a927b36-aff5-46d7-815b-7f994aa0f348 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:38.502179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df80a39-236f-4db0-a100-6264bdb82f3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:38.504780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df80a39-236f-4db0-a100-6264bdb82f3b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-2df80a39-236f-4db0-a100-6264bdb82f3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:38.522563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:38.523677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:38.524755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:38.524755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:38.524755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:38.524755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:38.524755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:38.524755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:38.524755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:38.524755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:38.527706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 298/607] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:38.528875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df80a39-236f-4db0-a100-6264bdb82f3b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:38.620418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a927b36-aff5-46d7-815b-7f994aa0f348 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:38.674081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab65682-4d6c-4da4-a5a6-f80617763fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:38.677820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab65682-4d6c-4da4-a5a6-f80617763fa2 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:38.678283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab65682-4d6c-4da4-a5a6-f80617763fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:38.678722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab65682-4d6c-4da4-a5a6-f80617763fa2 None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/316ad1c2eccf4642a21c68856585ceee/roles/941f412208c04a7abd395240a0339304` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:38.696601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab65682-4d6c-4da4-a5a6-f80617763fa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=316ad1c2eccf4642a21c68856585ceee, role_id=941f412208c04a7abd395240a0339304)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:38.698368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab65682-4d6c-4da4-a5a6-f80617763fa2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:38.698975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df80a39-236f-4db0-a100-6264bdb82f3b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:38.717580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:38.748103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df513259-7b86-4321-be83-1cd73275a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:38.748909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df513259-7b86-4321-be83-1cd73275a8bb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:38.749199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df513259-7b86-4321-be83-1cd73275a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:38.749486 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df513259-7b86-4321-be83-1cd73275a8bb None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:38.750248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df513259-7b86-4321-be83-1cd73275a8bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:38.750594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df513259-7b86-4321-be83-1cd73275a8bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:38.751973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df513259-7b86-4321-be83-1cd73275a8bb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:38.761029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 299/608] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:38.762518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df513259-7b86-4321-be83-1cd73275a8bb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:38.822933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 310/609] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:02:38 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/316ad1c2eccf4642a21c68856585ceee/roles/941f412208c04a7abd395240a0339304 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:02:38.824141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab65682-4d6c-4da4-a5a6-f80617763fa2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:38.897977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:38.969221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df513259-7b86-4321-be83-1cd73275a8bb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:38.985972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab65682-4d6c-4da4-a5a6-f80617763fa2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:39.050856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f49d6-42e1-4178-b90d-e953beb1bdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:39.052001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f49d6-42e1-4178-b90d-e953beb1bdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:39.052471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f49d6-42e1-4178-b90d-e953beb1bdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:39.054793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f49d6-42e1-4178-b90d-e953beb1bdb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:39.072124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc25424-7157-4da3-a7cf-d0820d8ca97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:39.073240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc25424-7157-4da3-a7cf-d0820d8ca97c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:39.073711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc25424-7157-4da3-a7cf-d0820d8ca97c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:39.074168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc25424-7157-4da3-a7cf-d0820d8ca97c None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:39.076311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc25424-7157-4da3-a7cf-d0820d8ca97c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:39.077093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc25424-7157-4da3-a7cf-d0820d8ca97c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:39.078842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc25424-7157-4da3-a7cf-d0820d8ca97c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:39.084183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f49d6-42e1-4178-b90d-e953beb1bdb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:39.085640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f49d6-42e1-4178-b90d-e953beb1bdb8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:39.097787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 300/610] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:39.102729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc25424-7157-4da3-a7cf-d0820d8ca97c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-334f49d6-42e1-4178-b90d-e953beb1bdb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:39.109133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:39.110059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:39.110997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:39.110997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:39.110997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:39.110997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:39.113725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 311/611] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:39.116365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f49d6-42e1-4178-b90d-e953beb1bdb8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:39.142805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:39.215602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:39.217999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc25424-7157-4da3-a7cf-d0820d8ca97c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:39.239904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f49d6-42e1-4178-b90d-e953beb1bdb8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:39.255211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07298ea0-6d65-4064-90d7-4f32c92a5a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:39.256472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07298ea0-6d65-4064-90d7-4f32c92a5a42 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:39.256980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07298ea0-6d65-4064-90d7-4f32c92a5a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:39.257460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07298ea0-6d65-4064-90d7-4f32c92a5a42 None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/8b266ea9964c4a179c2743b9cf6478eb/roles/941f412208c04a7abd395240a0339304` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:39.274223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07298ea0-6d65-4064-90d7-4f32c92a5a42 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=8b266ea9964c4a179c2743b9cf6478eb, role_id=941f412208c04a7abd395240a0339304)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:39.276031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07298ea0-6d65-4064-90d7-4f32c92a5a42 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:39.298127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f0c93-9819-430f-bc2d-751b9724aba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:39.299212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f0c93-9819-430f-bc2d-751b9724aba1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:39.299705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f0c93-9819-430f-bc2d-751b9724aba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:39.300188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f0c93-9819-430f-bc2d-751b9724aba1 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:39.301265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f0c93-9819-430f-bc2d-751b9724aba1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:39.301796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f0c93-9819-430f-bc2d-751b9724aba1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:39.303445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f0c93-9819-430f-bc2d-751b9724aba1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:39.316615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 312/612] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:39.321494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f0c93-9819-430f-bc2d-751b9724aba1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:39.375112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 301/613] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:02:39 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/8b266ea9964c4a179c2743b9cf6478eb/roles/941f412208c04a7abd395240a0339304 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:02:39.463784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f0c93-9819-430f-bc2d-751b9724aba1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:39.519629 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa17397-8a1c-4b75-89e9-08d7b6a1c145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:39.520352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa17397-8a1c-4b75-89e9-08d7b6a1c145 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:39.520638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa17397-8a1c-4b75-89e9-08d7b6a1c145 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:39.520913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa17397-8a1c-4b75-89e9-08d7b6a1c145 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:39.534151 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa17397-8a1c-4b75-89e9-08d7b6a1c145 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:39.535653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa17397-8a1c-4b75-89e9-08d7b6a1c145 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-cfa17397-8a1c-4b75-89e9-08d7b6a1c145 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:39.555795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:39.556987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:39.557915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 313/614] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:39.566719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07298ea0-6d65-4064-90d7-4f32c92a5a42 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:39.693522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:39.714343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07298ea0-6d65-4064-90d7-4f32c92a5a42 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:39.767632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34d5fd-f7b4-47a3-876e-9b0c5ec8d9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:39.768583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34d5fd-f7b4-47a3-876e-9b0c5ec8d9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:39.769068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34d5fd-f7b4-47a3-876e-9b0c5ec8d9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:39.769693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34d5fd-f7b4-47a3-876e-9b0c5ec8d9f7 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:39.770720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34d5fd-f7b4-47a3-876e-9b0c5ec8d9f7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:39.771408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34d5fd-f7b4-47a3-876e-9b0c5ec8d9f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:39.772990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34d5fd-f7b4-47a3-876e-9b0c5ec8d9f7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:39.803926 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 302/615] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 12:02:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:39.811334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa17397-8a1c-4b75-89e9-08d7b6a1c145 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:39.823583 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:39.990943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa17397-8a1c-4b75-89e9-08d7b6a1c145 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:40.071568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf34fb-ee78-4b9e-9a11-1fd77474e238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:40.072436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf34fb-ee78-4b9e-9a11-1fd77474e238 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:40.072720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf34fb-ee78-4b9e-9a11-1fd77474e238 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:40.072998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf34fb-ee78-4b9e-9a11-1fd77474e238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:40.092090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf34fb-ee78-4b9e-9a11-1fd77474e238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:40.093633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf34fb-ee78-4b9e-9a11-1fd77474e238 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ccaf34fb-ee78-4b9e-9a11-1fd77474e238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:40.122354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:40.123982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:40.125043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:40.125043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:40.125043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:40.125043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:40.125043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 314/616] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:40.131605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34d5fd-f7b4-47a3-876e-9b0c5ec8d9f7 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:40.191146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336de783-edeb-4d3b-a18b-9c2b0bd4cf29 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:40.191592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336de783-edeb-4d3b-a18b-9c2b0bd4cf29 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:40.192158 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336de783-edeb-4d3b-a18b-9c2b0bd4cf29 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:40.200144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 315/617] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:40.204497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc5114-a05b-47e4-88eb-9077856c94cb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:40.204948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc5114-a05b-47e4-88eb-9077856c94cb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:40.205336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc5114-a05b-47e4-88eb-9077856c94cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:40.231578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b3bc5114-a05b-47e4-88eb-9077856c94cb None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:40.233068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b3bc5114-a05b-47e4-88eb-9077856c94cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:40.330889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:40.347938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 316/618] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:40.386057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34d5fd-f7b4-47a3-876e-9b0c5ec8d9f7 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:40.427820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc5114-a05b-47e4-88eb-9077856c94cb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:40.465258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d04fd-271b-4325-b21f-b761122d26d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:40.466157 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d04fd-271b-4325-b21f-b761122d26d2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:40.466468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d04fd-271b-4325-b21f-b761122d26d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:40.466758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d04fd-271b-4325-b21f-b761122d26d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:40.467497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d04fd-271b-4325-b21f-b761122d26d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:40.467812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d04fd-271b-4325-b21f-b761122d26d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:40.470356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d04fd-271b-4325-b21f-b761122d26d2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:40.489443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:40.503677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 303/619] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:40.505973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d04fd-271b-4325-b21f-b761122d26d2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:40.653393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc5114-a05b-47e4-88eb-9077856c94cb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:40.722811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d04fd-271b-4325-b21f-b761122d26d2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:40.737582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b9987-016d-4272-843a-6fc4c0513f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:40.738422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b9987-016d-4272-843a-6fc4c0513f20 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:40.738718 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b9987-016d-4272-843a-6fc4c0513f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:40.738999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b9987-016d-4272-843a-6fc4c0513f20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:40.764330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b9987-016d-4272-843a-6fc4c0513f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-aa2b9987-016d-4272-843a-6fc4c0513f20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:40.782339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:40.783389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:40.784234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:40.784954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 317/620] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:40.794216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b9987-016d-4272-843a-6fc4c0513f20 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:40.810741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af872f8-05d0-4779-92c7-0c1329a12d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:40.811628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af872f8-05d0-4779-92c7-0c1329a12d6d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:40.812120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af872f8-05d0-4779-92c7-0c1329a12d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:40.812657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af872f8-05d0-4779-92c7-0c1329a12d6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:40.823884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af872f8-05d0-4779-92c7-0c1329a12d6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:40.827137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af872f8-05d0-4779-92c7-0c1329a12d6d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-7af872f8-05d0-4779-92c7-0c1329a12d6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:40.846364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:40.847365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:40.848233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:40.848233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:40.848233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:40.848233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:40.848233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:40.848233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:40.848233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:40.848233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:40.848233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 304/621] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:40.856829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af872f8-05d0-4779-92c7-0c1329a12d6d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:40.914857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:40.972027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:41.030205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b9987-016d-4272-843a-6fc4c0513f20 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:41.038809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af872f8-05d0-4779-92c7-0c1329a12d6d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:41.080295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e59fe7-88bf-49f2-b1fd-42d09e5cd70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:41.080914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e59fe7-88bf-49f2-b1fd-42d09e5cd70c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.081204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e59fe7-88bf-49f2-b1fd-42d09e5cd70c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.081480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e59fe7-88bf-49f2-b1fd-42d09e5cd70c None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.082237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e59fe7-88bf-49f2-b1fd-42d09e5cd70c None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:41.082581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e59fe7-88bf-49f2-b1fd-42d09e5cd70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:41.083960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e59fe7-88bf-49f2-b1fd-42d09e5cd70c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:41.085375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afc3d5-4b6a-4187-af12-861ad48e202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:41.086358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afc3d5-4b6a-4187-af12-861ad48e202c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.086856 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afc3d5-4b6a-4187-af12-861ad48e202c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.087404 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afc3d5-4b6a-4187-af12-861ad48e202c None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.088393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afc3d5-4b6a-4187-af12-861ad48e202c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:41.088903 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afc3d5-4b6a-4187-af12-861ad48e202c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:41.091344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afc3d5-4b6a-4187-af12-861ad48e202c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:41.093610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 318/622] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:41.099325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532e79-c1ce-445a-8bdb-4afc6832290d None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.099620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532e79-c1ce-445a-8bdb-4afc6832290d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.099913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532e79-c1ce-445a-8bdb-4afc6832290d None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.101644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 319/623] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:41.102401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532e79-c1ce-445a-8bdb-4afc6832290d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:41.107614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 305/624] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:41.109212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8965-2780-4862-8544-35165c452171 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.109815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8965-2780-4862-8544-35165c452171 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.110294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed8965-2780-4862-8544-35165c452171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.137861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8fed8965-2780-4862-8544-35165c452171 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:41.139764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8fed8965-2780-4862-8544-35165c452171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:41.230524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 306/625] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:41.230524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8965-2780-4862-8544-35165c452171 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:41.246111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532e79-c1ce-445a-8bdb-4afc6832290d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:41.266615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:41.287277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:41.295591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b168c26-ac84-4a89-892e-9d6063a74d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:41.296476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b168c26-ac84-4a89-892e-9d6063a74d11 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.296835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b168c26-ac84-4a89-892e-9d6063a74d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.297196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b168c26-ac84-4a89-892e-9d6063a74d11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.308178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b168c26-ac84-4a89-892e-9d6063a74d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:41.309762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b168c26-ac84-4a89-892e-9d6063a74d11 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-7b168c26-ac84-4a89-892e-9d6063a74d11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:41.329176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:41.330282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:41.331298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:41.331298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:41.331298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:41.331298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:41.334503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 320/626] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:41.335402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b168c26-ac84-4a89-892e-9d6063a74d11 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:41.355849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed8965-2780-4862-8544-35165c452171 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:41.406445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40883694-b46d-4ca8-83fb-6e106d77cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:41.408673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40883694-b46d-4ca8-83fb-6e106d77cddf None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.409113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40883694-b46d-4ca8-83fb-6e106d77cddf None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.409782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40883694-b46d-4ca8-83fb-6e106d77cddf None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/28ddf898ad7b4e10aaf0770249b4453d/roles/941f412208c04a7abd395240a0339304` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.430378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40883694-b46d-4ca8-83fb-6e106d77cddf None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=28ddf898ad7b4e10aaf0770249b4453d, role_id=941f412208c04a7abd395240a0339304)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:41.432444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40883694-b46d-4ca8-83fb-6e106d77cddf None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:41.453342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b168c26-ac84-4a89-892e-9d6063a74d11 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:41.490679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526678f6-0ceb-4037-9a31-846a984fdca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:41.491397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526678f6-0ceb-4037-9a31-846a984fdca6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.491685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526678f6-0ceb-4037-9a31-846a984fdca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.491965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526678f6-0ceb-4037-9a31-846a984fdca6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.503073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526678f6-0ceb-4037-9a31-846a984fdca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-526678f6-0ceb-4037-9a31-846a984fdca6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:41.510895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:41.512162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 321/627] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:41.513215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526678f6-0ceb-4037-9a31-846a984fdca6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:41.523558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 307/628] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:02:41 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/28ddf898ad7b4e10aaf0770249b4453d/roles/941f412208c04a7abd395240a0339304 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:02:41.525254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40883694-b46d-4ca8-83fb-6e106d77cddf None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:41.541661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:41.640463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526678f6-0ceb-4037-9a31-846a984fdca6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:41.673982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:41.678015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40883694-b46d-4ca8-83fb-6e106d77cddf None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:41.693074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab991094-0204-4552-babb-1c5b7babe074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:41.694571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab991094-0204-4552-babb-1c5b7babe074 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.695332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab991094-0204-4552-babb-1c5b7babe074 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.695862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab991094-0204-4552-babb-1c5b7babe074 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.696976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab991094-0204-4552-babb-1c5b7babe074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:41.697859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab991094-0204-4552-babb-1c5b7babe074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:41.699828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab991094-0204-4552-babb-1c5b7babe074 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:41.718032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080b154-3625-469a-9021-fbc137a398d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:41.718676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 322/629] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:41.720234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d2de7-b8e9-44fb-8ed1-fc422a7c432f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.720726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d2de7-b8e9-44fb-8ed1-fc422a7c432f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.721199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d2de7-b8e9-44fb-8ed1-fc422a7c432f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.723048 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 323/630] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:41.723803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080b154-3625-469a-9021-fbc137a398d7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.724096 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080b154-3625-469a-9021-fbc137a398d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.724302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d2de7-b8e9-44fb-8ed1-fc422a7c432f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:41.724435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080b154-3625-469a-9021-fbc137a398d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.732035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080b154-3625-469a-9021-fbc137a398d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:41.732035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080b154-3625-469a-9021-fbc137a398d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:41.732035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080b154-3625-469a-9021-fbc137a398d7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:41.738592 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 308/631] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:41.740033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17207d9-e555-4593-943a-11a3a1a7317a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.740281 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17207d9-e555-4593-943a-11a3a1a7317a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.740559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17207d9-e555-4593-943a-11a3a1a7317a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.760511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b17207d9-e555-4593-943a-11a3a1a7317a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:41.762214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b17207d9-e555-4593-943a-11a3a1a7317a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:41.836780 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:41.842445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 309/632] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:41.842445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17207d9-e555-4593-943a-11a3a1a7317a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:41.888582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d2de7-b8e9-44fb-8ed1-fc422a7c432f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:41.938508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf58360-2702-46c3-b606-d58b26e4e9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:41.940363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf58360-2702-46c3-b606-d58b26e4e9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:41.941298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf58360-2702-46c3-b606-d58b26e4e9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:41.942073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf58360-2702-46c3-b606-d58b26e4e9c4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:41.951815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:41.954784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf58360-2702-46c3-b606-d58b26e4e9c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:41.957042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf58360-2702-46c3-b606-d58b26e4e9c4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:41.961111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17207d9-e555-4593-943a-11a3a1a7317a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-baf58360-2702-46c3-b606-d58b26e4e9c4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:41.976360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:41.977355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 12:02:41.978292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:41.985104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 324/633] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:41.985104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf58360-2702-46c3-b606-d58b26e4e9c4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:42.006428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8add-439c-4707-8e98-765bbed8d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:42.007224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8add-439c-4707-8e98-765bbed8d1dd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.007509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8add-439c-4707-8e98-765bbed8d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.007784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8add-439c-4707-8e98-765bbed8d1dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.022841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8add-439c-4707-8e98-765bbed8d1dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:42.024309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8add-439c-4707-8e98-765bbed8d1dd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-d9eb8add-439c-4707-8e98-765bbed8d1dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:42.046210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:42.047185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:42.048065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:42.048065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:42.048065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:42.048065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:42.051936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 310/634] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:42.052681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8add-439c-4707-8e98-765bbed8d1dd None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:42.129767 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf58360-2702-46c3-b606-d58b26e4e9c4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:42.177294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f217a0b-ff36-450d-94fb-22908ca8bf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:42.179120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f217a0b-ff36-450d-94fb-22908ca8bf16 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.179876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f217a0b-ff36-450d-94fb-22908ca8bf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.180354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f217a0b-ff36-450d-94fb-22908ca8bf16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.181425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8add-439c-4707-8e98-765bbed8d1dd None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:42.191444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:42.196981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f217a0b-ff36-450d-94fb-22908ca8bf16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-3f217a0b-ff36-450d-94fb-22908ca8bf16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:42.208187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:42.219396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 325/635] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:42.220216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f217a0b-ff36-450d-94fb-22908ca8bf16 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:42.228402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aff84f-d3b5-4af0-ab29-33b6cb8bf42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:42.229517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aff84f-d3b5-4af0-ab29-33b6cb8bf42c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.229809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aff84f-d3b5-4af0-ab29-33b6cb8bf42c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.230090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aff84f-d3b5-4af0-ab29-33b6cb8bf42c None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.231099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aff84f-d3b5-4af0-ab29-33b6cb8bf42c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:42.231419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aff84f-d3b5-4af0-ab29-33b6cb8bf42c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:42.233253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aff84f-d3b5-4af0-ab29-33b6cb8bf42c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:42.254154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 311/636] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 12:02:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:42.255408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aff84f-d3b5-4af0-ab29-33b6cb8bf42c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:42.266435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:42.332561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f217a0b-ff36-450d-94fb-22908ca8bf16 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:42.368351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fb67a-13dc-4da4-b1ce-698c08e19c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:42.369014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fb67a-13dc-4da4-b1ce-698c08e19c00 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.369298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fb67a-13dc-4da4-b1ce-698c08e19c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.369572 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fb67a-13dc-4da4-b1ce-698c08e19c00 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.370022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aff84f-d3b5-4af0-ab29-33b6cb8bf42c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:42.370417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fb67a-13dc-4da4-b1ce-698c08e19c00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:42.370743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fb67a-13dc-4da4-b1ce-698c08e19c00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:42.372858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135fb67a-13dc-4da4-b1ce-698c08e19c00 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:42.384897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 326/637] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:42.385744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fb67a-13dc-4da4-b1ce-698c08e19c00 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:42.419767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc09b51-8285-4dbf-9ff3-66ddab52888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:42.420442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09b51-8285-4dbf-9ff3-66ddab52888a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.420720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09b51-8285-4dbf-9ff3-66ddab52888a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.420997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc09b51-8285-4dbf-9ff3-66ddab52888a None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.421826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc09b51-8285-4dbf-9ff3-66ddab52888a None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:42.422164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc09b51-8285-4dbf-9ff3-66ddab52888a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:42.423605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc09b51-8285-4dbf-9ff3-66ddab52888a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:42.432871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 312/638] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:42.434178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7c41d-302b-4666-85ed-b5607aa61bfd None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.434496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7c41d-302b-4666-85ed-b5607aa61bfd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.434778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7c41d-302b-4666-85ed-b5607aa61bfd None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.438517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 313/639] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:42.438517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7c41d-302b-4666-85ed-b5607aa61bfd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:42.457787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:42.507816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:42.508177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fb67a-13dc-4da4-b1ce-698c08e19c00 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:42.553248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7c41d-302b-4666-85ed-b5607aa61bfd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:42.556224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40423-c639-4c59-bfff-b685e65e9447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:42.556630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40423-c639-4c59-bfff-b685e65e9447 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.556919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40423-c639-4c59-bfff-b685e65e9447 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.557197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40423-c639-4c59-bfff-b685e65e9447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.590088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589f677-4b37-40c4-b60f-a08839308632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:42.590860 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589f677-4b37-40c4-b60f-a08839308632 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.591143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589f677-4b37-40c4-b60f-a08839308632 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.591417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589f677-4b37-40c4-b60f-a08839308632 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.600047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589f677-4b37-40c4-b60f-a08839308632 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:42.601543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589f677-4b37-40c4-b60f-a08839308632 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5589f677-4b37-40c4-b60f-a08839308632 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:42.616490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:42.623427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 314/640] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 12:02:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:42.624455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589f677-4b37-40c4-b60f-a08839308632 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:42.638008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c40423-c639-4c59-bfff-b685e65e9447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:42.639559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c40423-c639-4c59-bfff-b685e65e9447 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:42.700489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:42.725642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589f677-4b37-40c4-b60f-a08839308632 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:42.747955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 327/641] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:42.749624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ef824-1a9c-4747-be59-04193beb6260 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.750429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ef824-1a9c-4747-be59-04193beb6260 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.750755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ef824-1a9c-4747-be59-04193beb6260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.755766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:42.766447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283366c-c856-4b1a-9daa-de6bdbdd098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:42.769935 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283366c-c856-4b1a-9daa-de6bdbdd098e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.769935 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283366c-c856-4b1a-9daa-de6bdbdd098e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.769935 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283366c-c856-4b1a-9daa-de6bdbdd098e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.770245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-663ef824-1a9c-4747-be59-04193beb6260 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:42.770245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-663ef824-1a9c-4747-be59-04193beb6260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:42.782559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283366c-c856-4b1a-9daa-de6bdbdd098e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:42.784013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283366c-c856-4b1a-9daa-de6bdbdd098e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5283366c-c856-4b1a-9daa-de6bdbdd098e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:42.804341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:42.805535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:42.806429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:42.806429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:42.806429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:42.806429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:42.814103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 315/642] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:42.814103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283366c-c856-4b1a-9daa-de6bdbdd098e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:42.856757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 328/643] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:42.857901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ef824-1a9c-4747-be59-04193beb6260 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:42.935033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283366c-c856-4b1a-9daa-de6bdbdd098e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:42.958514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:42.968440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ef824-1a9c-4747-be59-04193beb6260 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:42.974255 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c244-d519-45c1-b079-d68272d6d30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:42.974949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c244-d519-45c1-b079-d68272d6d30c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:42.975230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c244-d519-45c1-b079-d68272d6d30c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:42.975503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c244-d519-45c1-b079-d68272d6d30c None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:42.976332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86c244-d519-45c1-b079-d68272d6d30c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:42.977040 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86c244-d519-45c1-b079-d68272d6d30c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:42.978072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86c244-d519-45c1-b079-d68272d6d30c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:42.995692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 316/644] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 12:02:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:42.996552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c244-d519-45c1-b079-d68272d6d30c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.010041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e024694-0f55-41aa-ae3b-3cc94830f1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.011159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e024694-0f55-41aa-ae3b-3cc94830f1af None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.012318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e024694-0f55-41aa-ae3b-3cc94830f1af None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.012840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e024694-0f55-41aa-ae3b-3cc94830f1af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.027062 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e024694-0f55-41aa-ae3b-3cc94830f1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:43.029293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e024694-0f55-41aa-ae3b-3cc94830f1af None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:43.044603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6e024694-0f55-41aa-ae3b-3cc94830f1af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:43.048965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:43.049996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:43.062539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:43.062539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:43.062539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:43.062539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:43.062539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 329/645] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:43.062539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e024694-0f55-41aa-ae3b-3cc94830f1af None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.123908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c244-d519-45c1-b079-d68272d6d30c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:43.161238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e024694-0f55-41aa-ae3b-3cc94830f1af None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:43.163502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35793a-fc14-4d52-9b7a-084d3c72a213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.164381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35793a-fc14-4d52-9b7a-084d3c72a213 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.164495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35793a-fc14-4d52-9b7a-084d3c72a213 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.164702 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35793a-fc14-4d52-9b7a-084d3c72a213 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.165423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35793a-fc14-4d52-9b7a-084d3c72a213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:43.165727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35793a-fc14-4d52-9b7a-084d3c72a213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:43.167364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35793a-fc14-4d52-9b7a-084d3c72a213 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:43.185451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 317/646] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:43.185451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35793a-fc14-4d52-9b7a-084d3c72a213 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.204030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef742b9-f6da-4003-9f41-58555fd52f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.204828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef742b9-f6da-4003-9f41-58555fd52f63 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.205113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef742b9-f6da-4003-9f41-58555fd52f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.205389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef742b9-f6da-4003-9f41-58555fd52f63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.215805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef742b9-f6da-4003-9f41-58555fd52f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-fef742b9-f6da-4003-9f41-58555fd52f63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:43.223492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:43.224431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:43.225265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 330/647] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:43.225946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef742b9-f6da-4003-9f41-58555fd52f63 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.228673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:43.292160 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:43.300498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35793a-fc14-4d52-9b7a-084d3c72a213 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:43.334047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621db3e-b6a7-4455-a135-ea07ad1ac6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.334750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621db3e-b6a7-4455-a135-ea07ad1ac6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.335035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621db3e-b6a7-4455-a135-ea07ad1ac6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.335311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621db3e-b6a7-4455-a135-ea07ad1ac6a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.335775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef742b9-f6da-4003-9f41-58555fd52f63 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:43.375807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823f9d4-2f29-49af-b848-4091bada9ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.376393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823f9d4-2f29-49af-b848-4091bada9ede None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.376675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823f9d4-2f29-49af-b848-4091bada9ede None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.376950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823f9d4-2f29-49af-b848-4091bada9ede None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.377672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823f9d4-2f29-49af-b848-4091bada9ede None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:43.377983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823f9d4-2f29-49af-b848-4091bada9ede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:43.379381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823f9d4-2f29-49af-b848-4091bada9ede None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:43.396806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 331/648] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:43.396806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823f9d4-2f29-49af-b848-4091bada9ede None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.414860 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621db3e-b6a7-4455-a135-ea07ad1ac6a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:43.416429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621db3e-b6a7-4455-a135-ea07ad1ac6a4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:43.464640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:43.508936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823f9d4-2f29-49af-b848-4091bada9ede None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:43.518925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:43.526484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 318/649] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:43.528408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621db3e-b6a7-4455-a135-ea07ad1ac6a4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.550304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab7f7e-abf6-494f-b979-908194dd71fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.551411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab7f7e-abf6-494f-b979-908194dd71fc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.551895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab7f7e-abf6-494f-b979-908194dd71fc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.552351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab7f7e-abf6-494f-b979-908194dd71fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.561577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab7f7e-abf6-494f-b979-908194dd71fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:43.563372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab7f7e-abf6-494f-b979-908194dd71fc None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d1ab7f7e-abf6-494f-b979-908194dd71fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:43.578889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:43.580073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 12:02:43.581259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:43.584581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 332/650] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:43.585290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab7f7e-abf6-494f-b979-908194dd71fc None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.641874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621db3e-b6a7-4455-a135-ea07ad1ac6a4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:43.683865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9e658-c259-45a0-85a8-905df045944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.684541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9e658-c259-45a0-85a8-905df045944d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.684829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9e658-c259-45a0-85a8-905df045944d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.685271 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9e658-c259-45a0-85a8-905df045944d None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.685853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9e658-c259-45a0-85a8-905df045944d None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:43.686165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9e658-c259-45a0-85a8-905df045944d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:43.687595 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9e658-c259-45a0-85a8-905df045944d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:43.688231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab7f7e-abf6-494f-b979-908194dd71fc None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:43.700690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 319/651] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:43.700690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9e658-c259-45a0-85a8-905df045944d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.716080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:43.727803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427821e-182e-4864-8d9b-00a589664d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.731181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427821e-182e-4864-8d9b-00a589664d03 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.731181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427821e-182e-4864-8d9b-00a589664d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.731181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427821e-182e-4864-8d9b-00a589664d03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.737511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427821e-182e-4864-8d9b-00a589664d03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:43.739037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427821e-182e-4864-8d9b-00a589664d03 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8427821e-182e-4864-8d9b-00a589664d03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:43.756506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:43.757505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:43.758384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:43.758384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:43.758384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:43.758384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:43.758384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:43.758384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:43.758384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:43.758384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:43.758384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 333/652] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:43.758931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427821e-182e-4864-8d9b-00a589664d03 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.771289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:43.816800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9e658-c259-45a0-85a8-905df045944d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:43.853146 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369d20b-9355-4601-b638-22674550fcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.854670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369d20b-9355-4601-b638-22674550fcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.855167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369d20b-9355-4601-b638-22674550fcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.856197 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369d20b-9355-4601-b638-22674550fcf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.865374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427821e-182e-4864-8d9b-00a589664d03 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:43.869673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369d20b-9355-4601-b638-22674550fcf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:43.871246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369d20b-9355-4601-b638-22674550fcf0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-f369d20b-9355-4601-b638-22674550fcf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:43.889611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:43.890670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:43.891564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:43.891564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:43.891564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:43.891564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:43.891564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 320/653] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:43.891845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369d20b-9355-4601-b638-22674550fcf0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.907248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef55b0-1e3a-4fe8-9d47-b8cbab58175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:43.908083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef55b0-1e3a-4fe8-9d47-b8cbab58175b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:43.908468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef55b0-1e3a-4fe8-9d47-b8cbab58175b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:43.908827 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef55b0-1e3a-4fe8-9d47-b8cbab58175b None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:43.909742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef55b0-1e3a-4fe8-9d47-b8cbab58175b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:43.910171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef55b0-1e3a-4fe8-9d47-b8cbab58175b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:43.911701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef55b0-1e3a-4fe8-9d47-b8cbab58175b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:43.931120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 334/654] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:43.932053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef55b0-1e3a-4fe8-9d47-b8cbab58175b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:43.973920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:43.998601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369d20b-9355-4601-b638-22674550fcf0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.007443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:44.033019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a96a1-2896-4753-b0a2-05d10de1da53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.033575 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a96a1-2896-4753-b0a2-05d10de1da53 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.033861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a96a1-2896-4753-b0a2-05d10de1da53 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.034139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a96a1-2896-4753-b0a2-05d10de1da53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.035964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef55b0-1e3a-4fe8-9d47-b8cbab58175b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.045194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a96a1-2896-4753-b0a2-05d10de1da53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:44.046670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a96a1-2896-4753-b0a2-05d10de1da53 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5a8a96a1-2896-4753-b0a2-05d10de1da53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:44.065010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:44.066092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:44.067125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:44.067125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:44.067125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:44.067125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:44.070719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 321/655] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:44.071478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a96a1-2896-4753-b0a2-05d10de1da53 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.071584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67603fdc-67cb-4e2f-962c-8900cea3f607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.072194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67603fdc-67cb-4e2f-962c-8900cea3f607 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.072484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67603fdc-67cb-4e2f-962c-8900cea3f607 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.072765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67603fdc-67cb-4e2f-962c-8900cea3f607 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.073552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67603fdc-67cb-4e2f-962c-8900cea3f607 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:44.073889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67603fdc-67cb-4e2f-962c-8900cea3f607 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:44.075887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67603fdc-67cb-4e2f-962c-8900cea3f607 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:44.087902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 335/656] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:44.089306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d057e6-b4fc-4a5c-bf69-47aa4f4ec209 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.089637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d057e6-b4fc-4a5c-bf69-47aa4f4ec209 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.089930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d057e6-b4fc-4a5c-bf69-47aa4f4ec209 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.092224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 336/657] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:44.092224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d057e6-b4fc-4a5c-bf69-47aa4f4ec209 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.180034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a96a1-2896-4753-b0a2-05d10de1da53 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.195699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:44.200285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d057e6-b4fc-4a5c-bf69-47aa4f4ec209 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.217346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa28b6-2413-4dab-b69b-d1a9eccc64a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.218136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa28b6-2413-4dab-b69b-d1a9eccc64a1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.218688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa28b6-2413-4dab-b69b-d1a9eccc64a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.219154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa28b6-2413-4dab-b69b-d1a9eccc64a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.219977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa28b6-2413-4dab-b69b-d1a9eccc64a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:44.220391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa28b6-2413-4dab-b69b-d1a9eccc64a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:44.221858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa28b6-2413-4dab-b69b-d1a9eccc64a1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:44.236544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 322/658] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:44.236852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e8ebb-f41c-418b-97d6-32207b605966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.237363 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa28b6-2413-4dab-b69b-d1a9eccc64a1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.237545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e8ebb-f41c-418b-97d6-32207b605966 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.237795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e8ebb-f41c-418b-97d6-32207b605966 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.238076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e8ebb-f41c-418b-97d6-32207b605966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.260457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:44.308394 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e8ebb-f41c-418b-97d6-32207b605966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:44.310010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e8ebb-f41c-418b-97d6-32207b605966 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:44.340943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa28b6-2413-4dab-b69b-d1a9eccc64a1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.375009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b6bfa-d4de-4db1-bc08-7788e5c42d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.375668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b6bfa-d4de-4db1-bc08-7788e5c42d53 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.375949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b6bfa-d4de-4db1-bc08-7788e5c42d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.376225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b6bfa-d4de-4db1-bc08-7788e5c42d53 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.376981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b6bfa-d4de-4db1-bc08-7788e5c42d53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:44.377285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b6bfa-d4de-4db1-bc08-7788e5c42d53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:44.378682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b6bfa-d4de-4db1-bc08-7788e5c42d53 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:44.390317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 323/659] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:44.391438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b6bfa-d4de-4db1-bc08-7788e5c42d53 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.408539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:44.412282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 337/660] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:44.415775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff42df-d4ba-4ce8-bdd4-7e737396334c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.416067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff42df-d4ba-4ce8-bdd4-7e737396334c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.416381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff42df-d4ba-4ce8-bdd4-7e737396334c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.434196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a6ff42df-d4ba-4ce8-bdd4-7e737396334c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:44.435334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a6ff42df-d4ba-4ce8-bdd4-7e737396334c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:44.474526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:44.493648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b6bfa-d4de-4db1-bc08-7788e5c42d53 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.502671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 338/661] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:44.503571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff42df-d4ba-4ce8-bdd4-7e737396334c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.532813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176751d0-33ed-4cb9-92fb-2782fc76436e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.533496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176751d0-33ed-4cb9-92fb-2782fc76436e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.533785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176751d0-33ed-4cb9-92fb-2782fc76436e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.534070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176751d0-33ed-4cb9-92fb-2782fc76436e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.535010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176751d0-33ed-4cb9-92fb-2782fc76436e None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=9969632e796345fa9671a45cf61ede3e) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:44.535324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176751d0-33ed-4cb9-92fb-2782fc76436e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:44.536739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176751d0-33ed-4cb9-92fb-2782fc76436e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:44.543936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 324/662] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/role_assignments?user.id=9969632e796345fa9671a45cf61ede3e&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:44.551213 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176751d0-33ed-4cb9-92fb-2782fc76436e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.630601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff42df-d4ba-4ce8-bdd4-7e737396334c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.663536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176751d0-33ed-4cb9-92fb-2782fc76436e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.663756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:44.671048 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae305d-943f-4fdd-94f5-35d5d315c66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.671849 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae305d-943f-4fdd-94f5-35d5d315c66e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.672130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae305d-943f-4fdd-94f5-35d5d315c66e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.680912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae305d-943f-4fdd-94f5-35d5d315c66e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.680912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae305d-943f-4fdd-94f5-35d5d315c66e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:44.681757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae305d-943f-4fdd-94f5-35d5d315c66e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8aae305d-943f-4fdd-94f5-35d5d315c66e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:44.697000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:44.698108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 339/663] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:44.699388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae305d-943f-4fdd-94f5-35d5d315c66e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.707960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769c446-fb12-4c2a-87db-b4cf2780aef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.708785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769c446-fb12-4c2a-87db-b4cf2780aef9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.709063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769c446-fb12-4c2a-87db-b4cf2780aef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.709341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769c446-fb12-4c2a-87db-b4cf2780aef9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.718470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769c446-fb12-4c2a-87db-b4cf2780aef9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:44.719916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769c446-fb12-4c2a-87db-b4cf2780aef9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0769c446-fb12-4c2a-87db-b4cf2780aef9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:44.739094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:44.740470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 325/664] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:44.741570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769c446-fb12-4c2a-87db-b4cf2780aef9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.747426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:44.823553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae305d-943f-4fdd-94f5-35d5d315c66e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.858923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769c446-fb12-4c2a-87db-b4cf2780aef9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:44.859161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f27ad-ecfd-4245-bbb9-0f8b5a91be3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.859882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f27ad-ecfd-4245-bbb9-0f8b5a91be3e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.860160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f27ad-ecfd-4245-bbb9-0f8b5a91be3e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.860431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f27ad-ecfd-4245-bbb9-0f8b5a91be3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.872226 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f27ad-ecfd-4245-bbb9-0f8b5a91be3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:44.873666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f27ad-ecfd-4245-bbb9-0f8b5a91be3e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-609f27ad-ecfd-4245-bbb9-0f8b5a91be3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:44.892957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:44.894283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:44.895269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:44.895269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:44.895269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:44.895269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:44.895269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 340/665] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:44.895553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f27ad-ecfd-4245-bbb9-0f8b5a91be3e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.901583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000f30e-d214-4308-9a6a-4cae4742236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:44.903776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000f30e-d214-4308-9a6a-4cae4742236b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:44.904228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000f30e-d214-4308-9a6a-4cae4742236b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:44.904685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000f30e-d214-4308-9a6a-4cae4742236b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:44.917814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000f30e-d214-4308-9a6a-4cae4742236b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4000f30e-d214-4308-9a6a-4cae4742236b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:44.926126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:44.927270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:44.928298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:44.930731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 326/666] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:44.931679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000f30e-d214-4308-9a6a-4cae4742236b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:44.951156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:44.998937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:45.011092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f27ad-ecfd-4245-bbb9-0f8b5a91be3e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.043969 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000f30e-d214-4308-9a6a-4cae4742236b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.051906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746f6a5-b7d3-49b3-8a65-a2b1aaec897b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.052653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746f6a5-b7d3-49b3-8a65-a2b1aaec897b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.053002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746f6a5-b7d3-49b3-8a65-a2b1aaec897b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.053292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746f6a5-b7d3-49b3-8a65-a2b1aaec897b None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.054087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746f6a5-b7d3-49b3-8a65-a2b1aaec897b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:45.054430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746f6a5-b7d3-49b3-8a65-a2b1aaec897b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:45.055888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746f6a5-b7d3-49b3-8a65-a2b1aaec897b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:45.076846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 341/667] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:45.076846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746f6a5-b7d3-49b3-8a65-a2b1aaec897b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:45.086499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd6782-470b-446d-8168-390dcf640c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.087171 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd6782-470b-446d-8168-390dcf640c20 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.087455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd6782-470b-446d-8168-390dcf640c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.087718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd6782-470b-446d-8168-390dcf640c20 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.088555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd6782-470b-446d-8168-390dcf640c20 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:45.088864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd6782-470b-446d-8168-390dcf640c20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:45.090918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd6782-470b-446d-8168-390dcf640c20 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:45.115179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 327/668] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 19 12:02:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:45.115878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd6782-470b-446d-8168-390dcf640c20 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:45.182637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746f6a5-b7d3-49b3-8a65-a2b1aaec897b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.184616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:45.221052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a2def-b61e-442c-9956-ded96c06b3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.221653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a2def-b61e-442c-9956-ded96c06b3fa None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.221977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a2def-b61e-442c-9956-ded96c06b3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.222213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a2def-b61e-442c-9956-ded96c06b3fa None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.222964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a2def-b61e-442c-9956-ded96c06b3fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:45.223269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a2def-b61e-442c-9956-ded96c06b3fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:45.224721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a2def-b61e-442c-9956-ded96c06b3fa None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:45.228274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd6782-470b-446d-8168-390dcf640c20 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.238043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 342/669] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:45.243097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a2def-b61e-442c-9956-ded96c06b3fa None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:45.253406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:45.265768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee1afc-717c-448e-9e22-f0903cabe227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.269912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee1afc-717c-448e-9e22-f0903cabe227 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.269912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee1afc-717c-448e-9e22-f0903cabe227 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.269912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee1afc-717c-448e-9e22-f0903cabe227 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.269912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee1afc-717c-448e-9e22-f0903cabe227 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:45.269912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee1afc-717c-448e-9e22-f0903cabe227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:45.269912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee1afc-717c-448e-9e22-f0903cabe227 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:45.278899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 328/670] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:45.279733 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee1afc-717c-448e-9e22-f0903cabe227 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:45.353402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a2def-b61e-442c-9956-ded96c06b3fa None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.385805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee1afc-717c-448e-9e22-f0903cabe227 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.388838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec2327-313b-4f6d-8ef6-990a7a6aabfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.389592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec2327-313b-4f6d-8ef6-990a7a6aabfa None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.389882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec2327-313b-4f6d-8ef6-990a7a6aabfa None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.390168 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfec2327-313b-4f6d-8ef6-990a7a6aabfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.391068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec2327-313b-4f6d-8ef6-990a7a6aabfa None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=bb6776311f354b748d563c4f1d793972) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:45.391392 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec2327-313b-4f6d-8ef6-990a7a6aabfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:45.392843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfec2327-313b-4f6d-8ef6-990a7a6aabfa None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:45.399165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 343/671] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:45 2022] GET /identity/v3/role_assignments?user.id=bb6776311f354b748d563c4f1d793972&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:45.399971 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec2327-313b-4f6d-8ef6-990a7a6aabfa None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:45.431882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:45.447169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d7fbc-ea69-4910-8373-d6a12fc86b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.448487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d7fbc-ea69-4910-8373-d6a12fc86b5d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.449564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d7fbc-ea69-4910-8373-d6a12fc86b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.450713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d7fbc-ea69-4910-8373-d6a12fc86b5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.522278 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfec2327-313b-4f6d-8ef6-990a7a6aabfa None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.535135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:45.557599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d7fbc-ea69-4910-8373-d6a12fc86b5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:45.559252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d7fbc-ea69-4910-8373-d6a12fc86b5d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:45.567150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94c082-d05a-42cc-8834-7978c3e2d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.568322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94c082-d05a-42cc-8834-7978c3e2d3ac None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.568783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94c082-d05a-42cc-8834-7978c3e2d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.569809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94c082-d05a-42cc-8834-7978c3e2d3ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.580308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94c082-d05a-42cc-8834-7978c3e2d3ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:45.582564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94c082-d05a-42cc-8834-7978c3e2d3ac None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9e94c082-d05a-42cc-8834-7978c3e2d3ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:45.603009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:45.604128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:45.606055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:45.606055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:45.606055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:45.606055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:45.606055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:45.606055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:45.606055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:45.606055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:45.611107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 344/672] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:45.613242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94c082-d05a-42cc-8834-7978c3e2d3ac None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:45.685623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 329/673] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:45.685623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d7fbc-ea69-4910-8373-d6a12fc86b5d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:45.721207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:45.736531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94c082-d05a-42cc-8834-7978c3e2d3ac None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.778829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618588e6-9cf5-471c-8a87-507128075b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.779917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618588e6-9cf5-471c-8a87-507128075b78 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.780307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618588e6-9cf5-471c-8a87-507128075b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.780716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618588e6-9cf5-471c-8a87-507128075b78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.795994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618588e6-9cf5-471c-8a87-507128075b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:45.798136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618588e6-9cf5-471c-8a87-507128075b78 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:45.798683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:45.811104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d7fbc-ea69-4910-8373-d6a12fc86b5d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-618588e6-9cf5-471c-8a87-507128075b78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:45.822217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:45.823214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:45.824710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:45.824710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:45.824710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:45.824710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:45.824710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 345/674] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:45.825616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618588e6-9cf5-471c-8a87-507128075b78 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:45.852872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4b5be-b0e6-4dcf-a278-515bdce4457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.853901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4b5be-b0e6-4dcf-a278-515bdce4457b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.854431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4b5be-b0e6-4dcf-a278-515bdce4457b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.854938 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4b5be-b0e6-4dcf-a278-515bdce4457b None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.856027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4b5be-b0e6-4dcf-a278-515bdce4457b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:45.856546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4b5be-b0e6-4dcf-a278-515bdce4457b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:45.858179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4b5be-b0e6-4dcf-a278-515bdce4457b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:45.870657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 330/675] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:45.872022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4b5be-b0e6-4dcf-a278-515bdce4457b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:45.954716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618588e6-9cf5-471c-8a87-507128075b78 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:45.997308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd7e01-98cb-42ab-b4a0-8791f75ddca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:45.998557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd7e01-98cb-42ab-b4a0-8791f75ddca3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:45.999058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd7e01-98cb-42ab-b4a0-8791f75ddca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:45.999513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd7e01-98cb-42ab-b4a0-8791f75ddca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:45.999920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4b5be-b0e6-4dcf-a278-515bdce4457b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:46.008810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:46.015021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd7e01-98cb-42ab-b4a0-8791f75ddca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:46.017929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd7e01-98cb-42ab-b4a0-8791f75ddca3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-45bd7e01-98cb-42ab-b4a0-8791f75ddca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:46.043729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:46.044812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:46.046109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:46.046109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:46.046109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:46.046109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:46.046109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 346/676] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:46.046109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd7e01-98cb-42ab-b4a0-8791f75ddca3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:46.056149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13920381-3345-436a-ae8b-2c548886a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:46.056931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13920381-3345-436a-ae8b-2c548886a902 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:46.057268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13920381-3345-436a-ae8b-2c548886a902 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:46.057551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13920381-3345-436a-ae8b-2c548886a902 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:46.059623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13920381-3345-436a-ae8b-2c548886a902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:46.059955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13920381-3345-436a-ae8b-2c548886a902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:46.061339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13920381-3345-436a-ae8b-2c548886a902 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:46.075446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 331/677] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:46.076285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13920381-3345-436a-ae8b-2c548886a902 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:46.080085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:46.186393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd7e01-98cb-42ab-b4a0-8791f75ddca3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:46.210093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13920381-3345-436a-ae8b-2c548886a902 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:46.222379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d31af1-3960-4d17-a1be-0fc12272bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:46.223395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d31af1-3960-4d17-a1be-0fc12272bc8c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:46.223954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d31af1-3960-4d17-a1be-0fc12272bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:46.224462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d31af1-3960-4d17-a1be-0fc12272bc8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:46.225484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d31af1-3960-4d17-a1be-0fc12272bc8c None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=316ad1c2eccf4642a21c68856585ceee) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:46.225995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d31af1-3960-4d17-a1be-0fc12272bc8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:46.227644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d31af1-3960-4d17-a1be-0fc12272bc8c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:46.233927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 347/678] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:46 2022] GET /identity/v3/role_assignments?user.id=316ad1c2eccf4642a21c68856585ceee&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:46.235101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d31af1-3960-4d17-a1be-0fc12272bc8c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:46.299829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ddc5-0c21-4108-b686-dd59a1130e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:46.300960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ddc5-0c21-4108-b686-dd59a1130e01 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:46.301460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ddc5-0c21-4108-b686-dd59a1130e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:46.301951 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ddc5-0c21-4108-b686-dd59a1130e01 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:46.303394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ddc5-0c21-4108-b686-dd59a1130e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:46.303776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ddc5-0c21-4108-b686-dd59a1130e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:46.305562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ddc5-0c21-4108-b686-dd59a1130e01 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:46.310853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:46.341021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 332/679] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:46.353854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ddc5-0c21-4108-b686-dd59a1130e01 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:46.365293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d31af1-3960-4d17-a1be-0fc12272bc8c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:46.405131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf7055-b9ba-409f-aa1d-fb0d063efc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:46.406302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf7055-b9ba-409f-aa1d-fb0d063efc41 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:46.406886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf7055-b9ba-409f-aa1d-fb0d063efc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:46.407358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf7055-b9ba-409f-aa1d-fb0d063efc41 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:46.417073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf7055-b9ba-409f-aa1d-fb0d063efc41 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:46.419943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf7055-b9ba-409f-aa1d-fb0d063efc41 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-deaf7055-b9ba-409f-aa1d-fb0d063efc41 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:46.436470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:46.437468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:46.438609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:46.444303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 348/680] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:46.446485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf7055-b9ba-409f-aa1d-fb0d063efc41 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:46.513504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:46.565448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ddc5-0c21-4108-b686-dd59a1130e01 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:46.608518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf7055-b9ba-409f-aa1d-fb0d063efc41 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:46.613279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698de1ed-51a6-4eb9-934a-830c8c78eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:46.614300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698de1ed-51a6-4eb9-934a-830c8c78eceb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:46.614845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698de1ed-51a6-4eb9-934a-830c8c78eceb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:46.615302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698de1ed-51a6-4eb9-934a-830c8c78eceb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:46.625247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:46.626167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698de1ed-51a6-4eb9-934a-830c8c78eceb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:46.627861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698de1ed-51a6-4eb9-934a-830c8c78eceb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-698de1ed-51a6-4eb9-934a-830c8c78eceb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:46.650085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:46.651059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:46.652586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:46.652586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:46.652586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:46.652586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:46.652586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:46.652586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:46.652586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:46.652586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:46.652586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 333/681] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:46.656093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b1513-b207-4801-bd5b-60f8a70e69ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:46.657074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b1513-b207-4801-bd5b-60f8a70e69ec None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:46.657580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b1513-b207-4801-bd5b-60f8a70e69ec None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:46.657787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698de1ed-51a6-4eb9-934a-830c8c78eceb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:46.658375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b1513-b207-4801-bd5b-60f8a70e69ec None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:46.659447 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b1513-b207-4801-bd5b-60f8a70e69ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:46.659975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b1513-b207-4801-bd5b-60f8a70e69ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:46.663031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b1513-b207-4801-bd5b-60f8a70e69ec None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:46.696286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 349/682] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 12:02:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:46.703442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b1513-b207-4801-bd5b-60f8a70e69ec None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:46.805056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698de1ed-51a6-4eb9-934a-830c8c78eceb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:46.852737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:46.877840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b1513-b207-4801-bd5b-60f8a70e69ec None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:46.880776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c07cc-a35b-4d44-8a19-bb38704e43c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:46.881467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c07cc-a35b-4d44-8a19-bb38704e43c3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:46.881747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c07cc-a35b-4d44-8a19-bb38704e43c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:46.882020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c07cc-a35b-4d44-8a19-bb38704e43c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:46.882882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c07cc-a35b-4d44-8a19-bb38704e43c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:46.883196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c07cc-a35b-4d44-8a19-bb38704e43c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:46.884747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c07cc-a35b-4d44-8a19-bb38704e43c3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:46.906855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 334/683] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:46.907947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c07cc-a35b-4d44-8a19-bb38704e43c3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:46.943734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311698-5c95-4097-87cf-0a4dd27ac24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:46.944531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311698-5c95-4097-87cf-0a4dd27ac24f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:46.944813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311698-5c95-4097-87cf-0a4dd27ac24f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:46.945087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311698-5c95-4097-87cf-0a4dd27ac24f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:47.016014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:47.068014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa311698-5c95-4097-87cf-0a4dd27ac24f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:47.069490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa311698-5c95-4097-87cf-0a4dd27ac24f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:47.079029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c07cc-a35b-4d44-8a19-bb38704e43c3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:47.114383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a374aee-051c-4c56-baae-ac210bdd23f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:47.115068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a374aee-051c-4c56-baae-ac210bdd23f4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:47.115353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a374aee-051c-4c56-baae-ac210bdd23f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:47.115625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a374aee-051c-4c56-baae-ac210bdd23f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:47.116463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a374aee-051c-4c56-baae-ac210bdd23f4 None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=8b266ea9964c4a179c2743b9cf6478eb) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:47.116779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a374aee-051c-4c56-baae-ac210bdd23f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:47.118211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a374aee-051c-4c56-baae-ac210bdd23f4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:47.126017 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 335/684] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:46 2022] GET /identity/v3/role_assignments?user.id=8b266ea9964c4a179c2743b9cf6478eb&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:47.131526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984b2ea-0e6a-4bf9-acf7-205f07aaec5b None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:47.131819 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984b2ea-0e6a-4bf9-acf7-205f07aaec5b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:47.132102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984b2ea-0e6a-4bf9-acf7-205f07aaec5b None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:47.133723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 336/685] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:47.139700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6715f-6a05-4cd8-a73a-725521fe50dc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:47.139989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6715f-6a05-4cd8-a73a-725521fe50dc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:47.140259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6715f-6a05-4cd8-a73a-725521fe50dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:47.171176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d9f6715f-6a05-4cd8-a73a-725521fe50dc None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:47.185019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d9f6715f-6a05-4cd8-a73a-725521fe50dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:47.185509 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 350/686] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:47.186857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311698-5c95-4097-87cf-0a4dd27ac24f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:47.282785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:47.286504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:47.340880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 337/687] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:47.345181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311698-5c95-4097-87cf-0a4dd27ac24f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:47.374219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6715f-6a05-4cd8-a73a-725521fe50dc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:47.393480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8413b-70c0-4fe8-9125-a4ffbb6831c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:47.394727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8413b-70c0-4fe8-9125-a4ffbb6831c6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:47.395228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8413b-70c0-4fe8-9125-a4ffbb6831c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:47.395714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8413b-70c0-4fe8-9125-a4ffbb6831c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:47.407718 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8413b-70c0-4fe8-9125-a4ffbb6831c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:47.409416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd8413b-70c0-4fe8-9125-a4ffbb6831c6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-bfd8413b-70c0-4fe8-9125-a4ffbb6831c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:47.428993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:47.429959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:47.430872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:47.430872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:47.430872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:47.430872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:47.430872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 351/688] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:47.436604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8413b-70c0-4fe8-9125-a4ffbb6831c6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:47.526578 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6715f-6a05-4cd8-a73a-725521fe50dc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:47.555992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8413b-70c0-4fe8-9125-a4ffbb6831c6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:47.574537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830efe3-62ce-40de-b8da-7c5ff7b48cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:47.575191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830efe3-62ce-40de-b8da-7c5ff7b48cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:47.575380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830efe3-62ce-40de-b8da-7c5ff7b48cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:47.575655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830efe3-62ce-40de-b8da-7c5ff7b48cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:47.583814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:47.588005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830efe3-62ce-40de-b8da-7c5ff7b48cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:47.589992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830efe3-62ce-40de-b8da-7c5ff7b48cf9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:47.596084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde0029-74da-48ff-8364-98761b43852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:47.596759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde0029-74da-48ff-8364-98761b43852f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:47.597085 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde0029-74da-48ff-8364-98761b43852f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:47.597618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde0029-74da-48ff-8364-98761b43852f None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:47.598159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde0029-74da-48ff-8364-98761b43852f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:47.598503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde0029-74da-48ff-8364-98761b43852f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:47.600021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde0029-74da-48ff-8364-98761b43852f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4830efe3-62ce-40de-b8da-7c5ff7b48cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:47.609697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:47.618946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:47.619834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:47.619834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:47.619834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:47.619834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:47.619834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 338/689] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:47.620102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 352/690] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:47.620182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830efe3-62ce-40de-b8da-7c5ff7b48cf9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:47.620251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde0029-74da-48ff-8364-98761b43852f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:47.659593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:47.733159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde0029-74da-48ff-8364-98761b43852f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:47.778929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458ba87-9172-4d48-b3ef-b23bc0779947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:47.781306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458ba87-9172-4d48-b3ef-b23bc0779947 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:47.781306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458ba87-9172-4d48-b3ef-b23bc0779947 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:47.781306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458ba87-9172-4d48-b3ef-b23bc0779947 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:47.786151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830efe3-62ce-40de-b8da-7c5ff7b48cf9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:47.793808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458ba87-9172-4d48-b3ef-b23bc0779947 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:47.795235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458ba87-9172-4d48-b3ef-b23bc0779947 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d458ba87-9172-4d48-b3ef-b23bc0779947 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:47.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:47.819268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:47.820165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:47.820165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:47.820165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:47.820165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:47.820165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:47.820165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:47.820165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:47.820165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:47.820165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 353/691] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:47.825263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458ba87-9172-4d48-b3ef-b23bc0779947 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:47.841950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281c9bb-245a-4e85-9ba2-d3d373c12be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:47.843045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281c9bb-245a-4e85-9ba2-d3d373c12be5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:47.845615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281c9bb-245a-4e85-9ba2-d3d373c12be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:47.847345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281c9bb-245a-4e85-9ba2-d3d373c12be5 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:47.849004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281c9bb-245a-4e85-9ba2-d3d373c12be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:47.850664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281c9bb-245a-4e85-9ba2-d3d373c12be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:47.853820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281c9bb-245a-4e85-9ba2-d3d373c12be5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:47.879797 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 339/692] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:47.885600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281c9bb-245a-4e85-9ba2-d3d373c12be5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:47.937592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:47.997353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:48.014754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281c9bb-245a-4e85-9ba2-d3d373c12be5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:48.072594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458ba87-9172-4d48-b3ef-b23bc0779947 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:48.103143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53824dea-6b6e-4fe3-802a-241eb22237f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:48.103832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53824dea-6b6e-4fe3-802a-241eb22237f9 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:48.104123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53824dea-6b6e-4fe3-802a-241eb22237f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:48.104406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53824dea-6b6e-4fe3-802a-241eb22237f9 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:48.105837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53824dea-6b6e-4fe3-802a-241eb22237f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:48.107399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53824dea-6b6e-4fe3-802a-241eb22237f9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:48.112564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb66fff-6572-46af-ba27-91944d628756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:48.113396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb66fff-6572-46af-ba27-91944d628756 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:48.113786 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb66fff-6572-46af-ba27-91944d628756 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:48.114166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb66fff-6572-46af-ba27-91944d628756 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:48.115150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb66fff-6572-46af-ba27-91944d628756 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:48.115570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb66fff-6572-46af-ba27-91944d628756 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:48.117101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb66fff-6572-46af-ba27-91944d628756 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:48.132865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 354/693] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:48.137656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e80744-5546-4c52-8ccf-88df416611b2 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:48.137656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e80744-5546-4c52-8ccf-88df416611b2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:48.137656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e80744-5546-4c52-8ccf-88df416611b2 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:48.137656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 355/694] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:48.139032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e80744-5546-4c52-8ccf-88df416611b2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:48.218741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 340/695] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:02:47 2022] POST /identity/v3/users => generated 254 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:02:48.218741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6b2d7-1649-4286-9337-d3dc4be5fb93 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:48.218741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6b2d7-1649-4286-9337-d3dc4be5fb93 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:48.218741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6b2d7-1649-4286-9337-d3dc4be5fb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:48.238369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-81e6b2d7-1649-4286-9337-d3dc4be5fb93 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:48.239718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-81e6b2d7-1649-4286-9337-d3dc4be5fb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:48.300207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e80744-5546-4c52-8ccf-88df416611b2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:48.305820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:48.334419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 341/696] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:48.364458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6b2d7-1649-4286-9337-d3dc4be5fb93 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:48.382078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8f4bc-31b5-4f5b-a81a-5fd9d9533344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:48.383283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8f4bc-31b5-4f5b-a81a-5fd9d9533344 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:48.383816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8f4bc-31b5-4f5b-a81a-5fd9d9533344 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:48.384295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8f4bc-31b5-4f5b-a81a-5fd9d9533344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:48.385513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8f4bc-31b5-4f5b-a81a-5fd9d9533344 None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=28ddf898ad7b4e10aaf0770249b4453d) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:48.386052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8f4bc-31b5-4f5b-a81a-5fd9d9533344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:48.389052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8f4bc-31b5-4f5b-a81a-5fd9d9533344 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:48.404714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 356/697] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:48 2022] GET /identity/v3/role_assignments?user.id=28ddf898ad7b4e10aaf0770249b4453d&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:48.488152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:48.525138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6b2d7-1649-4286-9337-d3dc4be5fb93 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:48.564764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269722b1-1e45-4152-b32d-18d4385df262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:48.565585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269722b1-1e45-4152-b32d-18d4385df262 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:48.565880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269722b1-1e45-4152-b32d-18d4385df262 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:48.566137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269722b1-1e45-4152-b32d-18d4385df262 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:48.578544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269722b1-1e45-4152-b32d-18d4385df262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-269722b1-1e45-4152-b32d-18d4385df262 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:48.586497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:48.587555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:48.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 342/698] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:48.594954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8f4bc-31b5-4f5b-a81a-5fd9d9533344 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:48.715676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8f4bc-31b5-4f5b-a81a-5fd9d9533344 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:48.762016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084033f-02fc-4cca-82c2-42b739a5449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:48.763595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084033f-02fc-4cca-82c2-42b739a5449d None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:48.764377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084033f-02fc-4cca-82c2-42b739a5449d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:48.765155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084033f-02fc-4cca-82c2-42b739a5449d None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:48.766651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084033f-02fc-4cca-82c2-42b739a5449d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:48.767207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084033f-02fc-4cca-82c2-42b739a5449d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:48.768828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084033f-02fc-4cca-82c2-42b739a5449d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:48.781232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 357/699] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:48.786483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269722b1-1e45-4152-b32d-18d4385df262 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:48.951608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269722b1-1e45-4152-b32d-18d4385df262 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:49.003152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7977435-e497-4337-b549-7fb823bb67ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:49.003966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7977435-e497-4337-b549-7fb823bb67ce None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:49.004252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7977435-e497-4337-b549-7fb823bb67ce None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:49.004530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7977435-e497-4337-b549-7fb823bb67ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:49.007689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:49.146984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7977435-e497-4337-b549-7fb823bb67ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:49.148449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7977435-e497-4337-b549-7fb823bb67ce None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c7977435-e497-4337-b549-7fb823bb67ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:49.172945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:49.173917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:49.175420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:49.175420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:49.175420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:49.175420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:49.175420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 343/700] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:49.179638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084033f-02fc-4cca-82c2-42b739a5449d None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:49.227050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:49.327579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084033f-02fc-4cca-82c2-42b739a5449d None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:49.364775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f437a46-bf5e-49a3-ad92-c6eb9a5938d6 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:49.365143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f437a46-bf5e-49a3-ad92-c6eb9a5938d6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:49.365396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f437a46-bf5e-49a3-ad92-c6eb9a5938d6 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:49.367067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 344/701] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:49.376462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9e68a-3ac1-4f2f-955f-612264136e25 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:49.376766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9e68a-3ac1-4f2f-955f-612264136e25 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:49.377019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9e68a-3ac1-4f2f-955f-612264136e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:49.384919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689d84d-e839-42b6-99c3-ca276791e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:49.385587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689d84d-e839-42b6-99c3-ca276791e2bc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:49.385870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689d84d-e839-42b6-99c3-ca276791e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:49.387081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689d84d-e839-42b6-99c3-ca276791e2bc None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:49.391931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689d84d-e839-42b6-99c3-ca276791e2bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:49.392248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689d84d-e839-42b6-99c3-ca276791e2bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:49.393712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689d84d-e839-42b6-99c3-ca276791e2bc None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:49.408089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a2c9e68a-3ac1-4f2f-955f-612264136e25 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:49.409220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a2c9e68a-3ac1-4f2f-955f-612264136e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:49.423566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 358/702] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:49.424497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689d84d-e839-42b6-99c3-ca276791e2bc None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:49.551407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 345/703] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:49.581360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9e68a-3ac1-4f2f-955f-612264136e25 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:49.608087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689d84d-e839-42b6-99c3-ca276791e2bc None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:49.610744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:49.680510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d42ee1-52c5-47cf-976e-3f05609d9079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:49.681388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d42ee1-52c5-47cf-976e-3f05609d9079 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:49.681722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d42ee1-52c5-47cf-976e-3f05609d9079 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:49.681997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d42ee1-52c5-47cf-976e-3f05609d9079 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:49.690456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9e68a-3ac1-4f2f-955f-612264136e25 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:49.693770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d42ee1-52c5-47cf-976e-3f05609d9079 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:49.695261 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d42ee1-52c5-47cf-976e-3f05609d9079 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:49.709485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:49.727795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace7d8b-4635-4da4-a581-cb47f9d3d618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:49.728828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace7d8b-4635-4da4-a581-cb47f9d3d618 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:49.729287 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace7d8b-4635-4da4-a581-cb47f9d3d618 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:49.729735 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace7d8b-4635-4da4-a581-cb47f9d3d618 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:49.731350 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace7d8b-4635-4da4-a581-cb47f9d3d618 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:49.732879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace7d8b-4635-4da4-a581-cb47f9d3d618 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-28d42ee1-52c5-47cf-976e-3f05609d9079 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:49.735328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:49.736693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:49.737841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 359/704] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 12:02:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:49.742223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d42ee1-52c5-47cf-976e-3f05609d9079 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-cace7d8b-4635-4da4-a581-cb47f9d3d618 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:49.752660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:49.753901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 12:02:49.754927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:49.757417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 346/705] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 12:02:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:49.764038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace7d8b-4635-4da4-a581-cb47f9d3d618 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:49.926498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d42ee1-52c5-47cf-976e-3f05609d9079 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:49.978880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace7d8b-4635-4da4-a581-cb47f9d3d618 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:50.016959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b3a94-44e8-4164-882e-efb238ace268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:50.019106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0177a7a-77f6-4be8-85a3-33336bc24354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:50.020055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0177a7a-77f6-4be8-85a3-33336bc24354 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.020510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0177a7a-77f6-4be8-85a3-33336bc24354 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.020998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0177a7a-77f6-4be8-85a3-33336bc24354 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.021890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0177a7a-77f6-4be8-85a3-33336bc24354 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:50.022369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0177a7a-77f6-4be8-85a3-33336bc24354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:50.023983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0177a7a-77f6-4be8-85a3-33336bc24354 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:50.029913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b3a94-44e8-4164-882e-efb238ace268 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.030293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b3a94-44e8-4164-882e-efb238ace268 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.030471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 347/706] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 12:02:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:50.032001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c7be-cf6f-4a39-824a-b13736e903b2 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.032478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c7be-cf6f-4a39-824a-b13736e903b2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.032907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c7be-cf6f-4a39-824a-b13736e903b2 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.034648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 348/707] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:50.035130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b3a94-44e8-4164-882e-efb238ace268 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.036429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b3a94-44e8-4164-882e-efb238ace268 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:50.036868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b3a94-44e8-4164-882e-efb238ace268 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:50.039170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b3a94-44e8-4164-882e-efb238ace268 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:50.040038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ac560-0e57-4204-948d-ad8218c0aeff None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.040523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ac560-0e57-4204-948d-ad8218c0aeff None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.041010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ac560-0e57-4204-948d-ad8218c0aeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.062023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-712ac560-0e57-4204-948d-ad8218c0aeff None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:50.063699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-712ac560-0e57-4204-948d-ad8218c0aeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:50.069820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 360/708] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 12:02:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:50.070984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b3a94-44e8-4164-882e-efb238ace268 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:50.083154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:50.113329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:50.150636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 349/709] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:50.151498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ac560-0e57-4204-948d-ad8218c0aeff None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:50.176016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b3a94-44e8-4164-882e-efb238ace268 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:50.213195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb1f94-376f-4163-ad7a-771ca474f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:50.213816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb1f94-376f-4163-ad7a-771ca474f3bb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.214240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb1f94-376f-4163-ad7a-771ca474f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.214615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb1f94-376f-4163-ad7a-771ca474f3bb None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.215655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb1f94-376f-4163-ad7a-771ca474f3bb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:50.215773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb1f94-376f-4163-ad7a-771ca474f3bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:50.217209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb1f94-376f-4163-ad7a-771ca474f3bb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:50.230568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 361/710] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 12:02:50 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:50.231692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb1f94-376f-4163-ad7a-771ca474f3bb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:50.271094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ac560-0e57-4204-948d-ad8218c0aeff None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:50.310074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348de37c-cb97-4804-8e2f-91c20ff0cda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:50.310762 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348de37c-cb97-4804-8e2f-91c20ff0cda1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.311046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348de37c-cb97-4804-8e2f-91c20ff0cda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.311326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348de37c-cb97-4804-8e2f-91c20ff0cda1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.368547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:50.374831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:50.384760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348de37c-cb97-4804-8e2f-91c20ff0cda1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:50.393957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348de37c-cb97-4804-8e2f-91c20ff0cda1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:50.454674 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb1f94-376f-4163-ad7a-771ca474f3bb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:50.496954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 350/711] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:50.503472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348de37c-cb97-4804-8e2f-91c20ff0cda1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:50.536041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfa292-99a5-44ef-9974-b2a5ca685576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:50.536981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfa292-99a5-44ef-9974-b2a5ca685576 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.537273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfa292-99a5-44ef-9974-b2a5ca685576 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.537552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfa292-99a5-44ef-9974-b2a5ca685576 None admin] PATH_INFO: `/v3/services/network` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.538219 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfa292-99a5-44ef-9974-b2a5ca685576 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:50.539832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cfa292-99a5-44ef-9974-b2a5ca685576 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-77cfa292-99a5-44ef-9974-b2a5ca685576 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:50.556721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:50.564515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:50.565523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 362/712] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:50 2022] GET /identity/v3/services/network => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:50.566304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfa292-99a5-44ef-9974-b2a5ca685576 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:50.598104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:50.632892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348de37c-cb97-4804-8e2f-91c20ff0cda1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:50.721696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa402f-151f-42bd-a36a-59ba223bdd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:50.722924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa402f-151f-42bd-a36a-59ba223bdd0e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.723350 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa402f-151f-42bd-a36a-59ba223bdd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.723763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa402f-151f-42bd-a36a-59ba223bdd0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.749868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa402f-151f-42bd-a36a-59ba223bdd0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:50.751288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa402f-151f-42bd-a36a-59ba223bdd0e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:50.753801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfa292-99a5-44ef-9974-b2a5ca685576 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1eaa402f-151f-42bd-a36a-59ba223bdd0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:50.789530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:50.790516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:50.791403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:50.791403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:50.791403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:50.791403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:50.795066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 351/713] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:50.800450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa402f-151f-42bd-a36a-59ba223bdd0e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:50.806279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60093e-3196-4285-9caa-e7e5d8fc84fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:50.807034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60093e-3196-4285-9caa-e7e5d8fc84fd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.807324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60093e-3196-4285-9caa-e7e5d8fc84fd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.807606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60093e-3196-4285-9caa-e7e5d8fc84fd None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.808545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60093e-3196-4285-9caa-e7e5d8fc84fd None admin] RBAC: Adding query filter params (name=network) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:50.808858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60093e-3196-4285-9caa-e7e5d8fc84fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:50.810260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60093e-3196-4285-9caa-e7e5d8fc84fd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:50.816371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:50.816622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 363/714] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:02:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:50.827533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60093e-3196-4285-9caa-e7e5d8fc84fd None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:50.928090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa402f-151f-42bd-a36a-59ba223bdd0e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:50.962963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:50.973691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff49144-76fc-4b93-8661-69ec2a040145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:50.974766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff49144-76fc-4b93-8661-69ec2a040145 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:50.975217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff49144-76fc-4b93-8661-69ec2a040145 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:50.978852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff49144-76fc-4b93-8661-69ec2a040145 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:50.978852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff49144-76fc-4b93-8661-69ec2a040145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:50.978852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff49144-76fc-4b93-8661-69ec2a040145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:50.981567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff49144-76fc-4b93-8661-69ec2a040145 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:51.008258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 352/715] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:51.014227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff49144-76fc-4b93-8661-69ec2a040145 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:51.043844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60093e-3196-4285-9caa-e7e5d8fc84fd None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:51.128482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b3c77-95b6-4f15-967d-77ec4430a04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:51.129167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b3c77-95b6-4f15-967d-77ec4430a04a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.129450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b3c77-95b6-4f15-967d-77ec4430a04a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.129729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b3c77-95b6-4f15-967d-77ec4430a04a None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.130463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b3c77-95b6-4f15-967d-77ec4430a04a None admin] RBAC: Adding query filter params (type=network) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:51.130778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b3c77-95b6-4f15-967d-77ec4430a04a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:51.132158 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b3c77-95b6-4f15-967d-77ec4430a04a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:51.147995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 364/716] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:02:50 2022] GET /identity/v3/services?type=network => generated 124 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:51.177207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833736f-f550-4bec-9ce9-02e06c05e178 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.177562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833736f-f550-4bec-9ce9-02e06c05e178 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.177762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833736f-f550-4bec-9ce9-02e06c05e178 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.179167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 365/717] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:51.185501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace2822-2df4-4c78-8318-07f61a176ba5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.185787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace2822-2df4-4c78-8318-07f61a176ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.186066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace2822-2df4-4c78-8318-07f61a176ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.211756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-aace2822-2df4-4c78-8318-07f61a176ba5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:51.213038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-aace2822-2df4-4c78-8318-07f61a176ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:51.243144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff49144-76fc-4b93-8661-69ec2a040145 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:51.313667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd193dbf-4023-47c3-83a3-7bf899608fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:51.314450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd193dbf-4023-47c3-83a3-7bf899608fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.314918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd193dbf-4023-47c3-83a3-7bf899608fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.318440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd193dbf-4023-47c3-83a3-7bf899608fd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.323383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd193dbf-4023-47c3-83a3-7bf899608fd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:51.325008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd193dbf-4023-47c3-83a3-7bf899608fd9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-dd193dbf-4023-47c3-83a3-7bf899608fd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:51.341420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:51.342621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:51.343723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:51.343723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:51.343723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:51.343723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:51.343723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:51.343723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:51.343723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:51.343723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:51.349892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 353/718] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:51.351279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd193dbf-4023-47c3-83a3-7bf899608fd9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:51.364081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:51.385741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 366/719] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 12:02:51.411548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace2822-2df4-4c78-8318-07f61a176ba5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:51.421044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:51.494728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd193dbf-4023-47c3-83a3-7bf899608fd9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:51.535379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace2822-2df4-4c78-8318-07f61a176ba5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:51.541217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177ab5c-4d04-4ca4-8cd7-4254ea5f2041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:51.542667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ab5c-4d04-4ca4-8cd7-4254ea5f2041 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.543178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ab5c-4d04-4ca4-8cd7-4254ea5f2041 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.544101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ab5c-4d04-4ca4-8cd7-4254ea5f2041 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.545116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177ab5c-4d04-4ca4-8cd7-4254ea5f2041 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:51.545603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177ab5c-4d04-4ca4-8cd7-4254ea5f2041 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:51.548130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177ab5c-4d04-4ca4-8cd7-4254ea5f2041 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:51.565235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 354/720] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:51.569408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d926-6211-4d48-88b6-50c64f09e478 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.569408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d926-6211-4d48-88b6-50c64f09e478 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.569408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d926-6211-4d48-88b6-50c64f09e478 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.569408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 355/721] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:51.572715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d926-6211-4d48-88b6-50c64f09e478 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:51.583338 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ded48-8ac4-47ee-baf1-52d50505a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:51.586142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ded48-8ac4-47ee-baf1-52d50505a63f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.586142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ded48-8ac4-47ee-baf1-52d50505a63f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.586142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ded48-8ac4-47ee-baf1-52d50505a63f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.596165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ded48-8ac4-47ee-baf1-52d50505a63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-099ded48-8ac4-47ee-baf1-52d50505a63f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:51.603977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:51.605868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:51.606763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 367/722] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:51.611385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc18ee-dd67-4270-a44a-713c3b93b259 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.611676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc18ee-dd67-4270-a44a-713c3b93b259 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.611958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc18ee-dd67-4270-a44a-713c3b93b259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.630191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-72dc18ee-dd67-4270-a44a-713c3b93b259 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:51.631393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-72dc18ee-dd67-4270-a44a-713c3b93b259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:51.693970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d926-6211-4d48-88b6-50c64f09e478 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:51.700399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:51.706831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:51.718398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 368/723] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:51.719529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc18ee-dd67-4270-a44a-713c3b93b259 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:51.740096 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c0bd-e4e4-4b8b-a66a-44786c70aeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:51.741533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c0bd-e4e4-4b8b-a66a-44786c70aeab None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.742286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c0bd-e4e4-4b8b-a66a-44786c70aeab None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.742750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c0bd-e4e4-4b8b-a66a-44786c70aeab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.744349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c0bd-e4e4-4b8b-a66a-44786c70aeab None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=bb6776311f354b748d563c4f1d793972) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:51.745099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c0bd-e4e4-4b8b-a66a-44786c70aeab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:51.747974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c0bd-e4e4-4b8b-a66a-44786c70aeab None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:51.756577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 356/724] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:51 2022] GET /identity/v3/role_assignments?user.id=bb6776311f354b748d563c4f1d793972&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=bd295cb5d1194b469beff0381c6b23af => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:51.767363 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c0bd-e4e4-4b8b-a66a-44786c70aeab None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:51.883247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc18ee-dd67-4270-a44a-713c3b93b259 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:51.934463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd2ced-9f36-435d-b887-bcf5f0acd6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:51.935163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd2ced-9f36-435d-b887-bcf5f0acd6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:51.935447 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd2ced-9f36-435d-b887-bcf5f0acd6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:51.936823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd2ced-9f36-435d-b887-bcf5f0acd6a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:51.937611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd2ced-9f36-435d-b887-bcf5f0acd6a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:51.937920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd2ced-9f36-435d-b887-bcf5f0acd6a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:51.939390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd2ced-9f36-435d-b887-bcf5f0acd6a0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:51.949257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 369/725] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:51.958432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd2ced-9f36-435d-b887-bcf5f0acd6a0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:51.981411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c0bd-e4e4-4b8b-a66a-44786c70aeab None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:52.024258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ff66-7f89-44d0-9810-e5e4e8d172a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:52.024258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ff66-7f89-44d0-9810-e5e4e8d172a4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:52.024258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ff66-7f89-44d0-9810-e5e4e8d172a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:52.024258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5ff66-7f89-44d0-9810-e5e4e8d172a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:52.050026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5ff66-7f89-44d0-9810-e5e4e8d172a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-14b5ff66-7f89-44d0-9810-e5e4e8d172a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:52.065667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:52.066789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:52.068067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 357/726] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:52.072883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:52.074713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ff66-7f89-44d0-9810-e5e4e8d172a4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:52.134076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:52.181689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd2ced-9f36-435d-b887-bcf5f0acd6a0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:52.241733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86081deb-ccc9-48fa-8162-228ca8449620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:52.242562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081deb-ccc9-48fa-8162-228ca8449620 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:52.243824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081deb-ccc9-48fa-8162-228ca8449620 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:52.246079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081deb-ccc9-48fa-8162-228ca8449620 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:52.262874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5ff66-7f89-44d0-9810-e5e4e8d172a4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:52.273183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86081deb-ccc9-48fa-8162-228ca8449620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:52.274663 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86081deb-ccc9-48fa-8162-228ca8449620 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-86081deb-ccc9-48fa-8162-228ca8449620 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:52.308411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:52.309379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:52.310260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:52.310260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:52.310260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:52.310260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:52.315189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 370/727] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:52.319470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86081deb-ccc9-48fa-8162-228ca8449620 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:52.339164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef02b6c-fb33-4d15-8723-f7d7dd1f397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:52.340207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef02b6c-fb33-4d15-8723-f7d7dd1f397e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:52.340719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef02b6c-fb33-4d15-8723-f7d7dd1f397e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:52.341222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef02b6c-fb33-4d15-8723-f7d7dd1f397e None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:52.342183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef02b6c-fb33-4d15-8723-f7d7dd1f397e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:52.342734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef02b6c-fb33-4d15-8723-f7d7dd1f397e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:52.344306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef02b6c-fb33-4d15-8723-f7d7dd1f397e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:52.359233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 358/728] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:52.364216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef02b6c-fb33-4d15-8723-f7d7dd1f397e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:52.509298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:52.514787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86081deb-ccc9-48fa-8162-228ca8449620 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:52.544624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:52.559492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef02b6c-fb33-4d15-8723-f7d7dd1f397e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:52.584336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bf2af-cc66-4c20-976c-95bf874f6923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:52.585073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf2af-cc66-4c20-976c-95bf874f6923 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:52.585361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf2af-cc66-4c20-976c-95bf874f6923 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:52.585636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bf2af-cc66-4c20-976c-95bf874f6923 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:52.586362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bf2af-cc66-4c20-976c-95bf874f6923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:52.586699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bf2af-cc66-4c20-976c-95bf874f6923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:52.588082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bf2af-cc66-4c20-976c-95bf874f6923 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:52.602908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 371/729] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:52.608744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bf2af-cc66-4c20-976c-95bf874f6923 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:52.630566 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c554b8-852c-41f3-91e9-8c47001b33fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:52.631371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c554b8-852c-41f3-91e9-8c47001b33fb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:52.631655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c554b8-852c-41f3-91e9-8c47001b33fb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:52.631929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c554b8-852c-41f3-91e9-8c47001b33fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:52.659611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c554b8-852c-41f3-91e9-8c47001b33fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:52.661070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c554b8-852c-41f3-91e9-8c47001b33fb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-a5c554b8-852c-41f3-91e9-8c47001b33fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:52.689800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:52.690795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:52.691669 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:52.691669 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:52.691669 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:52.691669 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:52.699963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 359/730] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:52.705297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c554b8-852c-41f3-91e9-8c47001b33fb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:52.721111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bf2af-cc66-4c20-976c-95bf874f6923 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:52.769132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13fbc3-3f26-4af9-932b-1fb4c0fb968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:52.769939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13fbc3-3f26-4af9-932b-1fb4c0fb968f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:52.770240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13fbc3-3f26-4af9-932b-1fb4c0fb968f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:52.770546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13fbc3-3f26-4af9-932b-1fb4c0fb968f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:52.786717 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13fbc3-3f26-4af9-932b-1fb4c0fb968f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:52.787481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13fbc3-3f26-4af9-932b-1fb4c0fb968f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-db13fbc3-3f26-4af9-932b-1fb4c0fb968f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:52.818096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:52.819138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:02:52.820069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:52.825531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 372/731] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:02:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:52.826807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c554b8-852c-41f3-91e9-8c47001b33fb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:52.831288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13fbc3-3f26-4af9-932b-1fb4c0fb968f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:52.867986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54b53c-0bac-4a71-ac87-3bc4b5662382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:52.869044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54b53c-0bac-4a71-ac87-3bc4b5662382 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:52.869480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:52.869764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54b53c-0bac-4a71-ac87-3bc4b5662382 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:52.870326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54b53c-0bac-4a71-ac87-3bc4b5662382 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:52.871545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54b53c-0bac-4a71-ac87-3bc4b5662382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:52.872086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54b53c-0bac-4a71-ac87-3bc4b5662382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:52.873668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54b53c-0bac-4a71-ac87-3bc4b5662382 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:52.887415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 360/732] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:52.899411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54b53c-0bac-4a71-ac87-3bc4b5662382 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:52.913487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:52.961765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13fbc3-3f26-4af9-932b-1fb4c0fb968f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:52.999411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4c72b-d895-4dc9-9e1e-5f153b7e4517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:53.000460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4c72b-d895-4dc9-9e1e-5f153b7e4517 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.001141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4c72b-d895-4dc9-9e1e-5f153b7e4517 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.001652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4c72b-d895-4dc9-9e1e-5f153b7e4517 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.002848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4c72b-d895-4dc9-9e1e-5f153b7e4517 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:53.003368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4c72b-d895-4dc9-9e1e-5f153b7e4517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:53.005077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4c72b-d895-4dc9-9e1e-5f153b7e4517 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:53.024492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 373/733] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 12:02:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:53.026364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55d0cb-1945-45df-9457-9c53d2f4388d None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.028247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55d0cb-1945-45df-9457-9c53d2f4388d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.028818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55d0cb-1945-45df-9457-9c53d2f4388d None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.030816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 374/734] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:53.032238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55d0cb-1945-45df-9457-9c53d2f4388d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:53.058749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54b53c-0bac-4a71-ac87-3bc4b5662382 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:53.096267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146afcbf-3700-49d0-99b3-07aea403a5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:53.097082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146afcbf-3700-49d0-99b3-07aea403a5ed None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.097356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146afcbf-3700-49d0-99b3-07aea403a5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.097642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146afcbf-3700-49d0-99b3-07aea403a5ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.106756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146afcbf-3700-49d0-99b3-07aea403a5ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:53.108321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146afcbf-3700-49d0-99b3-07aea403a5ed None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:53.120099 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-146afcbf-3700-49d0-99b3-07aea403a5ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:53.129349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:53.131086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:53.134580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:53.138908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 361/735] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:53.141031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b5777-f183-404d-a2ed-9880c0887585 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.141849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b5777-f183-404d-a2ed-9880c0887585 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.142919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b5777-f183-404d-a2ed-9880c0887585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.154559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55d0cb-1945-45df-9457-9c53d2f4388d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:53.165687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-dc3b5777-f183-404d-a2ed-9880c0887585 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:53.167786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dc3b5777-f183-404d-a2ed-9880c0887585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:53.203415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e415a-08a4-4270-9242-f0612ed80911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:53.204072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e415a-08a4-4270-9242-f0612ed80911 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.204374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e415a-08a4-4270-9242-f0612ed80911 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.204624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e415a-08a4-4270-9242-f0612ed80911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.260865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:53.281494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 362/736] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:53.281494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b5777-f183-404d-a2ed-9880c0887585 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:53.283017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e415a-08a4-4270-9242-f0612ed80911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:53.284682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e415a-08a4-4270-9242-f0612ed80911 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:53.373953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:53.400645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 375/737] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:53.403510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e415a-08a4-4270-9242-f0612ed80911 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:53.407021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b5777-f183-404d-a2ed-9880c0887585 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:53.446891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b56ff-c22d-473b-978b-7dd72ec102dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:53.447720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b56ff-c22d-473b-978b-7dd72ec102dc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.448076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b56ff-c22d-473b-978b-7dd72ec102dc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.449003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b56ff-c22d-473b-978b-7dd72ec102dc None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.449920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b56ff-c22d-473b-978b-7dd72ec102dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:53.450304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b56ff-c22d-473b-978b-7dd72ec102dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:53.451836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b56ff-c22d-473b-978b-7dd72ec102dc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:53.476219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 363/738] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 12:02:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:53.477343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b56ff-c22d-473b-978b-7dd72ec102dc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:53.519200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e415a-08a4-4270-9242-f0612ed80911 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:53.548902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:53.565408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d1af-c82c-4e40-8fb1-77ce2beb9a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:53.566399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d1af-c82c-4e40-8fb1-77ce2beb9a67 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.566876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d1af-c82c-4e40-8fb1-77ce2beb9a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.567324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1d1af-c82c-4e40-8fb1-77ce2beb9a67 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.568273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1d1af-c82c-4e40-8fb1-77ce2beb9a67 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:53.574009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1d1af-c82c-4e40-8fb1-77ce2beb9a67 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:53.595449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b56ff-c22d-473b-978b-7dd72ec102dc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:53.662370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233994ca-1beb-4435-a835-34091e3041f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:53.663227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233994ca-1beb-4435-a835-34091e3041f8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.663514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233994ca-1beb-4435-a835-34091e3041f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.663793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233994ca-1beb-4435-a835-34091e3041f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.676174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233994ca-1beb-4435-a835-34091e3041f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:53.677756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233994ca-1beb-4435-a835-34091e3041f8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:53.696939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 376/739] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:02:53 2022] POST /identity/v3/services => generated 251 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:02:53.698110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d1af-c82c-4e40-8fb1-77ce2beb9a67 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-233994ca-1beb-4435-a835-34091e3041f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:53.702955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:53.704170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:53.705086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:53.705086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:53.705086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:53.705086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:53.705086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 364/740] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:53.706123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233994ca-1beb-4435-a835-34091e3041f8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:53.759062 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:53.845079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1d1af-c82c-4e40-8fb1-77ce2beb9a67 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:53.896014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e462c-53d3-44f9-8fce-93ac0f59569a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:53.897379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e462c-53d3-44f9-8fce-93ac0f59569a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.898482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e462c-53d3-44f9-8fce-93ac0f59569a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.899041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e462c-53d3-44f9-8fce-93ac0f59569a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.921212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233994ca-1beb-4435-a835-34091e3041f8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:53.930746 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:53.975153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74331466-7fff-4bd2-86a5-d5d6a47ebdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:53.976137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74331466-7fff-4bd2-86a5-d5d6a47ebdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.976641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74331466-7fff-4bd2-86a5-d5d6a47ebdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.977119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74331466-7fff-4bd2-86a5-d5d6a47ebdd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.977855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74331466-7fff-4bd2-86a5-d5d6a47ebdd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:53.978169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74331466-7fff-4bd2-86a5-d5d6a47ebdd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:53.979620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74331466-7fff-4bd2-86a5-d5d6a47ebdd0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:53.993800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 365/741] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:53.994978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcb8a1-4138-4835-8ddc-9d59f0538fbb None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:53.995366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcb8a1-4138-4835-8ddc-9d59f0538fbb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:53.996312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcb8a1-4138-4835-8ddc-9d59f0538fbb None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:53.998988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 366/742] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:53.998988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcb8a1-4138-4835-8ddc-9d59f0538fbb None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:54.004454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e462c-53d3-44f9-8fce-93ac0f59569a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:54.005925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e462c-53d3-44f9-8fce-93ac0f59569a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:54.046434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:54.126291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcb8a1-4138-4835-8ddc-9d59f0538fbb None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:54.137074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 377/743] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:02:53 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 12:02:54.138851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb66155-6eea-4405-97ff-faba16343f31 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:54.139438 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb66155-6eea-4405-97ff-faba16343f31 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:54.139733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb66155-6eea-4405-97ff-faba16343f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:54.163987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4fb66155-6eea-4405-97ff-faba16343f31 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:54.165523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4fb66155-6eea-4405-97ff-faba16343f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:54.168936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6e1e7-d2d5-4cb2-be77-b3e63887c7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:54.169919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6e1e7-d2d5-4cb2-be77-b3e63887c7af None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:54.170209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6e1e7-d2d5-4cb2-be77-b3e63887c7af None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:54.170519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6e1e7-d2d5-4cb2-be77-b3e63887c7af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:54.179260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6e1e7-d2d5-4cb2-be77-b3e63887c7af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:54.180934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6e1e7-d2d5-4cb2-be77-b3e63887c7af None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-8fb6e1e7-d2d5-4cb2-be77-b3e63887c7af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:54.197976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:54.198991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:54.200406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:54.200406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:54.200406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:54.200406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:54.200406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:54.200406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:54.200406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:54.200406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:54.200406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 367/744] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:54.201583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6e1e7-d2d5-4cb2-be77-b3e63887c7af None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:54.232418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:54.260344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 378/745] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 12:02:54.261339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb66155-6eea-4405-97ff-faba16343f31 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:54.338454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6e1e7-d2d5-4cb2-be77-b3e63887c7af None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:54.346449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:54.375742 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631de72a-32fe-4756-a77b-c3510d7fc379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:54.376726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631de72a-32fe-4756-a77b-c3510d7fc379 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:54.376991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631de72a-32fe-4756-a77b-c3510d7fc379 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:54.377312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631de72a-32fe-4756-a77b-c3510d7fc379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:54.389474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb66155-6eea-4405-97ff-faba16343f31 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:54.392218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631de72a-32fe-4756-a77b-c3510d7fc379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:54.393758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631de72a-32fe-4756-a77b-c3510d7fc379 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-631de72a-32fe-4756-a77b-c3510d7fc379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:54.412425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:54.413502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:54.414378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:54.414378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:54.414378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:54.414378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:54.414378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 368/746] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:54.414378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631de72a-32fe-4756-a77b-c3510d7fc379 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:54.434508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6445442-9474-4a58-b006-6f9f27f75b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:54.435177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6445442-9474-4a58-b006-6f9f27f75b7f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:54.435576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6445442-9474-4a58-b006-6f9f27f75b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:54.435853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6445442-9474-4a58-b006-6f9f27f75b7f None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:54.436818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6445442-9474-4a58-b006-6f9f27f75b7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:54.437131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6445442-9474-4a58-b006-6f9f27f75b7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:54.438597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6445442-9474-4a58-b006-6f9f27f75b7f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:54.452146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 379/747] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:54.453773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80218c8-9afe-4f6c-b1d6-8d32debbab17 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:54.454064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80218c8-9afe-4f6c-b1d6-8d32debbab17 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:54.454344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80218c8-9afe-4f6c-b1d6-8d32debbab17 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:54.455798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 380/748] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:54.456289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80218c8-9afe-4f6c-b1d6-8d32debbab17 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:54.506736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:54.529788 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631de72a-32fe-4756-a77b-c3510d7fc379 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:54.565846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80218c8-9afe-4f6c-b1d6-8d32debbab17 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:54.570295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e484a-410f-4829-b955-f5f2e03b0de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:54.571104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e484a-410f-4829-b955-f5f2e03b0de2 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:54.571377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e484a-410f-4829-b955-f5f2e03b0de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:54.571652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e484a-410f-4829-b955-f5f2e03b0de2 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:54.572378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e484a-410f-4829-b955-f5f2e03b0de2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:54.574643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e484a-410f-4829-b955-f5f2e03b0de2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:54.643627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:54.655400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113da4d-ed78-46b7-8c0a-7fbc3a1e8932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:54.656068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113da4d-ed78-46b7-8c0a-7fbc3a1e8932 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:54.656399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113da4d-ed78-46b7-8c0a-7fbc3a1e8932 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:54.656682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113da4d-ed78-46b7-8c0a-7fbc3a1e8932 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:54.657404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113da4d-ed78-46b7-8c0a-7fbc3a1e8932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:54.657723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113da4d-ed78-46b7-8c0a-7fbc3a1e8932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:54.659126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113da4d-ed78-46b7-8c0a-7fbc3a1e8932 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:54.675000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 381/749] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:54.675736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113da4d-ed78-46b7-8c0a-7fbc3a1e8932 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:54.683264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 369/750] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:02:54 2022] POST /identity/v3/services => generated 240 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:02:54.684666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05706dc2-ebfb-4066-be83-ac4fb2fc3951 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:54.691925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05706dc2-ebfb-4066-be83-ac4fb2fc3951 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:54.691925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05706dc2-ebfb-4066-be83-ac4fb2fc3951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:54.709119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-05706dc2-ebfb-4066-be83-ac4fb2fc3951 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:54.710552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-05706dc2-ebfb-4066-be83-ac4fb2fc3951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:54.815534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113da4d-ed78-46b7-8c0a-7fbc3a1e8932 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:54.833497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 370/751] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:02:54.834687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05706dc2-ebfb-4066-be83-ac4fb2fc3951 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:54.856842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d218dbf-a208-46a4-a853-f6f7d5a366a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:54.857677 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d218dbf-a208-46a4-a853-f6f7d5a366a8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:54.858064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d218dbf-a208-46a4-a853-f6f7d5a366a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:54.858497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d218dbf-a208-46a4-a853-f6f7d5a366a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:54.859614 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d218dbf-a208-46a4-a853-f6f7d5a366a8 None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=6be447727ed44a48a6b023303289e319) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:54.860087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d218dbf-a208-46a4-a853-f6f7d5a366a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:54.861684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d218dbf-a208-46a4-a853-f6f7d5a366a8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:54.869340 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 382/752] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:54 2022] GET /identity/v3/role_assignments?user.id=6be447727ed44a48a6b023303289e319&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:54.871612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d218dbf-a208-46a4-a853-f6f7d5a366a8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:54.929856 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:54.961554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:54.985905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05706dc2-ebfb-4066-be83-ac4fb2fc3951 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:55.035683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d218dbf-a208-46a4-a853-f6f7d5a366a8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:55.046216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67448570-0a87-4385-a41d-16fb8b26c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:55.047123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448570-0a87-4385-a41d-16fb8b26c479 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:55.047408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448570-0a87-4385-a41d-16fb8b26c479 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:55.047687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67448570-0a87-4385-a41d-16fb8b26c479 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:55.059137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67448570-0a87-4385-a41d-16fb8b26c479 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:55.060731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67448570-0a87-4385-a41d-16fb8b26c479 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-67448570-0a87-4385-a41d-16fb8b26c479 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:55.079343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:55.080293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:55.081153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:55.081153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:55.081153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:55.081153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:55.081153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:55.081153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:55.081153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:55.081153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:55.083923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 371/753] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:55.090653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67448570-0a87-4385-a41d-16fb8b26c479 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:55.096018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10522952-023f-40d2-b9a4-1f026ea00e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:55.097195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10522952-023f-40d2-b9a4-1f026ea00e6f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:55.097682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10522952-023f-40d2-b9a4-1f026ea00e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:55.098175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10522952-023f-40d2-b9a4-1f026ea00e6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:55.115680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10522952-023f-40d2-b9a4-1f026ea00e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-10522952-023f-40d2-b9a4-1f026ea00e6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:55.124606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:55.125544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:55.126364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:55.129073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 383/754] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:55.134342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10522952-023f-40d2-b9a4-1f026ea00e6f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:55.260905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67448570-0a87-4385-a41d-16fb8b26c479 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:55.287962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:55.301978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37ef84-955f-47e4-a8e7-e50a1fafc862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:55.303765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37ef84-955f-47e4-a8e7-e50a1fafc862 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:55.303765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37ef84-955f-47e4-a8e7-e50a1fafc862 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:55.303765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37ef84-955f-47e4-a8e7-e50a1fafc862 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:55.304224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37ef84-955f-47e4-a8e7-e50a1fafc862 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:55.304542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37ef84-955f-47e4-a8e7-e50a1fafc862 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:55.305982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37ef84-955f-47e4-a8e7-e50a1fafc862 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:55.326558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 372/755] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:55.330208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37ef84-955f-47e4-a8e7-e50a1fafc862 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:55.382364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10522952-023f-40d2-b9a4-1f026ea00e6f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:55.390659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:55.428456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86370f24-5b8b-47c9-a429-c4ccafa4cea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:55.429588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86370f24-5b8b-47c9-a429-c4ccafa4cea3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:55.431062 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86370f24-5b8b-47c9-a429-c4ccafa4cea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:55.431588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86370f24-5b8b-47c9-a429-c4ccafa4cea3 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:55.432545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86370f24-5b8b-47c9-a429-c4ccafa4cea3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:55.433779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86370f24-5b8b-47c9-a429-c4ccafa4cea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:55.435381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86370f24-5b8b-47c9-a429-c4ccafa4cea3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:55.453150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 384/756] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:55.458051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86370f24-5b8b-47c9-a429-c4ccafa4cea3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:55.460459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37ef84-955f-47e4-a8e7-e50a1fafc862 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:55.501435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d093c4-ff8b-4c6c-b028-406f76047fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:55.502577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d093c4-ff8b-4c6c-b028-406f76047fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:55.502870 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d093c4-ff8b-4c6c-b028-406f76047fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:55.503199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d093c4-ff8b-4c6c-b028-406f76047fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:55.504047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d093c4-ff8b-4c6c-b028-406f76047fa9 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=28ddf898ad7b4e10aaf0770249b4453d) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:55.504369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d093c4-ff8b-4c6c-b028-406f76047fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:55.505816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d093c4-ff8b-4c6c-b028-406f76047fa9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:55.513223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 373/757] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:02:55 2022] GET /identity/v3/role_assignments?user.id=28ddf898ad7b4e10aaf0770249b4453d&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=bd295cb5d1194b469beff0381c6b23af => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:55.609300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86370f24-5b8b-47c9-a429-c4ccafa4cea3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:55.656498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927145e-0abc-4aff-817c-1bf22a513dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:55.657559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927145e-0abc-4aff-817c-1bf22a513dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:55.657991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927145e-0abc-4aff-817c-1bf22a513dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:55.658426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927145e-0abc-4aff-817c-1bf22a513dd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:55.676054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927145e-0abc-4aff-817c-1bf22a513dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:55.678060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927145e-0abc-4aff-817c-1bf22a513dd4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8927145e-0abc-4aff-817c-1bf22a513dd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:55.701204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:55.702366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:55.703469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:55.703469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:55.703469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:55.703469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:55.709333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 385/758] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:55.710710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d093c4-ff8b-4c6c-b028-406f76047fa9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:55.810517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:55.872140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d093c4-ff8b-4c6c-b028-406f76047fa9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:55.920483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612c1f0-7940-4f28-b4ab-41be892da19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:55.921156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c1f0-7940-4f28-b4ab-41be892da19d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:55.921434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c1f0-7940-4f28-b4ab-41be892da19d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:55.921710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c1f0-7940-4f28-b4ab-41be892da19d None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:55.922459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612c1f0-7940-4f28-b4ab-41be892da19d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:55.922774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612c1f0-7940-4f28-b4ab-41be892da19d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:55.924141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612c1f0-7940-4f28-b4ab-41be892da19d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:55.941749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 374/759] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:55.946825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927145e-0abc-4aff-817c-1bf22a513dd4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:55.982944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:56.083337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927145e-0abc-4aff-817c-1bf22a513dd4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:56.124043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec81aa-9380-4ad3-a772-c67d5ce8dc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:56.124848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec81aa-9380-4ad3-a772-c67d5ce8dc68 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:56.125136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec81aa-9380-4ad3-a772-c67d5ce8dc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:56.125409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec81aa-9380-4ad3-a772-c67d5ce8dc68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:56.134617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec81aa-9380-4ad3-a772-c67d5ce8dc68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:56.136221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec81aa-9380-4ad3-a772-c67d5ce8dc68 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-53ec81aa-9380-4ad3-a772-c67d5ce8dc68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:56.158013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:56.159055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 12:02:56.159984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:56.163372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 386/760] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 12:02:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:56.169682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612c1f0-7940-4f28-b4ab-41be892da19d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:56.335465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f01fc-8d64-47de-87b9-479ab8326a05 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:56.335660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f01fc-8d64-47de-87b9-479ab8326a05 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:56.335937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f01fc-8d64-47de-87b9-479ab8326a05 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:56.338519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 387/761] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:56.340897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612c1f0-7940-4f28-b4ab-41be892da19d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:56.346834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986301d-9bb4-4e7e-b8a0-26c0eab79195 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:56.347128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986301d-9bb4-4e7e-b8a0-26c0eab79195 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:56.347411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986301d-9bb4-4e7e-b8a0-26c0eab79195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:56.356252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:56.377585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c986301d-9bb4-4e7e-b8a0-26c0eab79195 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:56.378785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c986301d-9bb4-4e7e-b8a0-26c0eab79195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:56.406288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b518eff-2a6d-48be-9fb1-c9c86dafb351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:56.406996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b518eff-2a6d-48be-9fb1-c9c86dafb351 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:56.407275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b518eff-2a6d-48be-9fb1-c9c86dafb351 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:56.407543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b518eff-2a6d-48be-9fb1-c9c86dafb351 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:56.408385 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b518eff-2a6d-48be-9fb1-c9c86dafb351 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:56.408694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b518eff-2a6d-48be-9fb1-c9c86dafb351 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:56.410121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b518eff-2a6d-48be-9fb1-c9c86dafb351 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:56.453453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 375/762] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 12:02:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:56.458958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b518eff-2a6d-48be-9fb1-c9c86dafb351 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:56.493414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 388/763] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:02:56.530279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986301d-9bb4-4e7e-b8a0-26c0eab79195 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:56.546777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:56.655306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b518eff-2a6d-48be-9fb1-c9c86dafb351 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:56.682219 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986301d-9bb4-4e7e-b8a0-26c0eab79195 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:56.720535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74411e01-df7b-4efa-b95b-f9708dae97b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:56.721355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74411e01-df7b-4efa-b95b-f9708dae97b4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:56.721641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74411e01-df7b-4efa-b95b-f9708dae97b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:56.721921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74411e01-df7b-4efa-b95b-f9708dae97b4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:56.722606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74411e01-df7b-4efa-b95b-f9708dae97b4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:56.724157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74411e01-df7b-4efa-b95b-f9708dae97b4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-74411e01-df7b-4efa-b95b-f9708dae97b4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:56.736735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:56.737863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:56.738823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 389/764] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 12:02:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:56.745977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74411e01-df7b-4efa-b95b-f9708dae97b4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:56.760050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c4ea4-f6a1-46a7-a2cc-9dfa7c5716d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:56.761459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c4ea4-f6a1-46a7-a2cc-9dfa7c5716d5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:56.761755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c4ea4-f6a1-46a7-a2cc-9dfa7c5716d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:56.762026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c4ea4-f6a1-46a7-a2cc-9dfa7c5716d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:56.787490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c4ea4-f6a1-46a7-a2cc-9dfa7c5716d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:56.788973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c4ea4-f6a1-46a7-a2cc-9dfa7c5716d5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:56.831147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5e9c4ea4-f6a1-46a7-a2cc-9dfa7c5716d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:56.835297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:56.836262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:56.837156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:56.837156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:56.837156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:56.837156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:56.837156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 376/765] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:56.842958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c4ea4-f6a1-46a7-a2cc-9dfa7c5716d5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:56.858494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74411e01-df7b-4efa-b95b-f9708dae97b4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:56.888976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:56.901809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68c2bb-a915-46d3-9aba-3f099a33ac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:56.902730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68c2bb-a915-46d3-9aba-3f099a33ac14 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:56.903022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68c2bb-a915-46d3-9aba-3f099a33ac14 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:56.903304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68c2bb-a915-46d3-9aba-3f099a33ac14 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:56.904027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68c2bb-a915-46d3-9aba-3f099a33ac14 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:56.904338 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68c2bb-a915-46d3-9aba-3f099a33ac14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:56.905785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68c2bb-a915-46d3-9aba-3f099a33ac14 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:56.911789 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 390/766] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 12:02:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:56.920496 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68c2bb-a915-46d3-9aba-3f099a33ac14 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:57.063709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c4ea4-f6a1-46a7-a2cc-9dfa7c5716d5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:57.082089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68c2bb-a915-46d3-9aba-3f099a33ac14 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:57.108888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc710f-c294-4c7f-9d88-88a1ad748b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:57.109570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc710f-c294-4c7f-9d88-88a1ad748b72 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.109854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc710f-c294-4c7f-9d88-88a1ad748b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.110128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc710f-c294-4c7f-9d88-88a1ad748b72 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.111731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc710f-c294-4c7f-9d88-88a1ad748b72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:57.112046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc710f-c294-4c7f-9d88-88a1ad748b72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:57.113437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc710f-c294-4c7f-9d88-88a1ad748b72 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:57.134933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 377/767] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:57.145332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc710f-c294-4c7f-9d88-88a1ad748b72 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:57.150702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57307675-496f-4361-84b8-5d923d0b4392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:57.151998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57307675-496f-4361-84b8-5d923d0b4392 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.152287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57307675-496f-4361-84b8-5d923d0b4392 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.152845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57307675-496f-4361-84b8-5d923d0b4392 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.153596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57307675-496f-4361-84b8-5d923d0b4392 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:57.153910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57307675-496f-4361-84b8-5d923d0b4392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:57.156363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57307675-496f-4361-84b8-5d923d0b4392 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:57.171145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 391/768] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 12:02:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:57.176899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57307675-496f-4361-84b8-5d923d0b4392 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:57.183406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:57.314765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:57.349451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc710f-c294-4c7f-9d88-88a1ad748b72 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:57.369102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57307675-496f-4361-84b8-5d923d0b4392 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:57.420985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38f0a3-edfc-4f4d-bc91-c977c15bd4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:57.421569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38f0a3-edfc-4f4d-bc91-c977c15bd4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.421851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38f0a3-edfc-4f4d-bc91-c977c15bd4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.422131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38f0a3-edfc-4f4d-bc91-c977c15bd4d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.423134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38f0a3-edfc-4f4d-bc91-c977c15bd4d3 None admin] RBAC: Adding query filter params (interface=public, service_id=6284c289c0984f38b5b0698a3ccf08e5, region_id=RegionOne) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:57.423458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38f0a3-edfc-4f4d-bc91-c977c15bd4d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:57.424988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38f0a3-edfc-4f4d-bc91-c977c15bd4d3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:57.432472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137fa4d-adb2-4716-9c54-f26cb37c544a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:57.433428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137fa4d-adb2-4716-9c54-f26cb37c544a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.433714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137fa4d-adb2-4716-9c54-f26cb37c544a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.433995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137fa4d-adb2-4716-9c54-f26cb37c544a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.443981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 392/769] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 12:02:57 2022] GET /identity/v3/endpoints?service_id=6284c289c0984f38b5b0698a3ccf08e5&interface=public®ion_id=RegionOne => generated 194 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:57.445026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137fa4d-adb2-4716-9c54-f26cb37c544a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:57.451646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38f0a3-edfc-4f4d-bc91-c977c15bd4d3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:57.454597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137fa4d-adb2-4716-9c54-f26cb37c544a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-a137fa4d-adb2-4716-9c54-f26cb37c544a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:02:57.480580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:57.481587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:57.482517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:02:57.482517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:02:57.482517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:02:57.482517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:02:57.482517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:02:57.482517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:02:57.482517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:02:57.482517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:57.483817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 378/770] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:57.491539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137fa4d-adb2-4716-9c54-f26cb37c544a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:57.561582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:57.584974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38f0a3-edfc-4f4d-bc91-c977c15bd4d3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:57.600604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137fa4d-adb2-4716-9c54-f26cb37c544a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:57.622360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363fb59e-a187-4a3d-9a56-0c5edd352760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:57.623059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fb59e-a187-4a3d-9a56-0c5edd352760 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.623460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fb59e-a187-4a3d-9a56-0c5edd352760 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.623745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363fb59e-a187-4a3d-9a56-0c5edd352760 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.624443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fb59e-a187-4a3d-9a56-0c5edd352760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:57.625831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363fb59e-a187-4a3d-9a56-0c5edd352760 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:57.632197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 393/771] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 12:02:57 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:57.633719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e51ec7-1b9e-499c-8444-0f868c9ffdd6 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.634081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e51ec7-1b9e-499c-8444-0f868c9ffdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.634366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e51ec7-1b9e-499c-8444-0f868c9ffdd6 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.641009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 394/772] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:57.643065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3363f-0412-4363-a351-8071cf1fa4fd None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.643373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3363f-0412-4363-a351-8071cf1fa4fd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.643670 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3363f-0412-4363-a351-8071cf1fa4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.647553 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d959eb5-4cb3-4096-b3db-14635d1a071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:57.647553 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d959eb5-4cb3-4096-b3db-14635d1a071b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.647553 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d959eb5-4cb3-4096-b3db-14635d1a071b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.647553 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d959eb5-4cb3-4096-b3db-14635d1a071b None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.647894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d959eb5-4cb3-4096-b3db-14635d1a071b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:57.647894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d959eb5-4cb3-4096-b3db-14635d1a071b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:57.649366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d959eb5-4cb3-4096-b3db-14635d1a071b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:57.669765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:57.670101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-92a3363f-0412-4363-a351-8071cf1fa4fd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:57.673547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-92a3363f-0412-4363-a351-8071cf1fa4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:57.677154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 379/773] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:02:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:57.684984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d959eb5-4cb3-4096-b3db-14635d1a071b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:57.813072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 395/774] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:02:57.855067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106520c-1279-48e7-99c5-c66533275185 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.855699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106520c-1279-48e7-99c5-c66533275185 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.856183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106520c-1279-48e7-99c5-c66533275185 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.858004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 396/775] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:02:57.868541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82652b1d-98dc-46ec-9316-baee033144c2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.869185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82652b1d-98dc-46ec-9316-baee033144c2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.869766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82652b1d-98dc-46ec-9316-baee033144c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.888925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-82652b1d-98dc-46ec-9316-baee033144c2 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:57.890133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-82652b1d-98dc-46ec-9316-baee033144c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:57.892358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d959eb5-4cb3-4096-b3db-14635d1a071b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:57.931760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe211dd-49cb-486b-b821-726dd37666da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:57.932979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe211dd-49cb-486b-b821-726dd37666da None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:57.933415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe211dd-49cb-486b-b821-726dd37666da None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:57.933882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe211dd-49cb-486b-b821-726dd37666da None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/bb6776311f354b748d563c4f1d793972/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:57.936591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe211dd-49cb-486b-b821-726dd37666da None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=bb6776311f354b748d563c4f1d793972, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:57.938750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe211dd-49cb-486b-b821-726dd37666da None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:57.953219 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:57.967841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 397/776] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:02:57.969107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82652b1d-98dc-46ec-9316-baee033144c2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:58.037389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 380/777] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:02:57 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/bb6776311f354b748d563c4f1d793972/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:02:58.038226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe211dd-49cb-486b-b821-726dd37666da None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:58.128813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:58.183344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe211dd-49cb-486b-b821-726dd37666da None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:58.187084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82652b1d-98dc-46ec-9316-baee033144c2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:58.224246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d031588-62eb-4798-9265-d878de23d7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:58.225583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d031588-62eb-4798-9265-d878de23d7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.226098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d031588-62eb-4798-9265-d878de23d7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.226673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d031588-62eb-4798-9265-d878de23d7d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.241979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d031588-62eb-4798-9265-d878de23d7d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:58.250019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65f163-e922-4e67-bdaf-9f72e211d52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:58.251150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65f163-e922-4e67-bdaf-9f72e211d52c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-2d031588-62eb-4798-9265-d878de23d7d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:58.251335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:58.252241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:58.256848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 381/778] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:02:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:58.257578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65f163-e922-4e67-bdaf-9f72e211d52c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.257578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65f163-e922-4e67-bdaf-9f72e211d52c None admin] PATH_INFO: `/v3/services/network` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.257578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65f163-e922-4e67-bdaf-9f72e211d52c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:58.257578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65f163-e922-4e67-bdaf-9f72e211d52c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:58.261869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d031588-62eb-4798-9265-d878de23d7d5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-7b65f163-e922-4e67-bdaf-9f72e211d52c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:58.283530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:58.284576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:58.286256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 398/779] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:57 2022] GET /identity/v3/services/network => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:58.287332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d204b-7280-41f0-83f4-59e66415122d None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.287332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d204b-7280-41f0-83f4-59e66415122d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.287332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d204b-7280-41f0-83f4-59e66415122d None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.288508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 399/780] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:02:58.291287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d204b-7280-41f0-83f4-59e66415122d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:58.360722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:58.409720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d031588-62eb-4798-9265-d878de23d7d5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:58.414832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d204b-7280-41f0-83f4-59e66415122d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:58.436554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:58.458728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288fc76-d37b-4c91-8662-fb80c759ccc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:58.459470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288fc76-d37b-4c91-8662-fb80c759ccc2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.459873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288fc76-d37b-4c91-8662-fb80c759ccc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.460228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288fc76-d37b-4c91-8662-fb80c759ccc2 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.461101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288fc76-d37b-4c91-8662-fb80c759ccc2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:58.461409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b7225-4ee9-4b18-a4c5-cde380115ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:58.461539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288fc76-d37b-4c91-8662-fb80c759ccc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:58.462459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b7225-4ee9-4b18-a4c5-cde380115ccd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.462921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b7225-4ee9-4b18-a4c5-cde380115ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.463247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288fc76-d37b-4c91-8662-fb80c759ccc2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:58.463555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b7225-4ee9-4b18-a4c5-cde380115ccd None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.465600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b7225-4ee9-4b18-a4c5-cde380115ccd None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:58.466119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b7225-4ee9-4b18-a4c5-cde380115ccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:58.468060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b7225-4ee9-4b18-a4c5-cde380115ccd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:58.469031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 400/781] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:02:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:58.470376 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d26a6-13ef-41ea-916f-9a2bd50fde8f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.470909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d26a6-13ef-41ea-916f-9a2bd50fde8f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.471289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d26a6-13ef-41ea-916f-9a2bd50fde8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.485163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 382/782] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:02:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:58.485163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b7225-4ee9-4b18-a4c5-cde380115ccd None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:58.492769 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-545d26a6-13ef-41ea-916f-9a2bd50fde8f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:02:58.493963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-545d26a6-13ef-41ea-916f-9a2bd50fde8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:02:58.578729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 401/783] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:02:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:02:58.580015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d26a6-13ef-41ea-916f-9a2bd50fde8f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:58.625354 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b7225-4ee9-4b18-a4c5-cde380115ccd None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:58.647768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:58.670307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d455811-5dcf-4708-a017-736885b9d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:58.671586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d455811-5dcf-4708-a017-736885b9d123 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.672098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d455811-5dcf-4708-a017-736885b9d123 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.672582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d455811-5dcf-4708-a017-736885b9d123 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.673502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d455811-5dcf-4708-a017-736885b9d123 None admin] RBAC: Adding query filter params (type=network) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:58.674022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d455811-5dcf-4708-a017-736885b9d123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:58.675657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d455811-5dcf-4708-a017-736885b9d123 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:58.684840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 383/784] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:02:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:58.686242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d455811-5dcf-4708-a017-736885b9d123 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:58.702984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d26a6-13ef-41ea-916f-9a2bd50fde8f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:58.738989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23d108-f339-4947-aef2-76e1609d6d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:58.739775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d108-f339-4947-aef2-76e1609d6d38 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.740059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d108-f339-4947-aef2-76e1609d6d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.740326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d108-f339-4947-aef2-76e1609d6d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.740778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:58.751861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23d108-f339-4947-aef2-76e1609d6d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:58.754175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23d108-f339-4947-aef2-76e1609d6d38 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ff23d108-f339-4947-aef2-76e1609d6d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:58.775469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:58.777098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:58.777979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:58.777979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:58.777979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:58.777979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:58.782186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 402/785] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:58.783178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23d108-f339-4947-aef2-76e1609d6d38 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:58.811794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d455811-5dcf-4708-a017-736885b9d123 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:58.850725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b441499-fb38-42f7-b7b5-08c17bfbda38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:58.851837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b441499-fb38-42f7-b7b5-08c17bfbda38 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.852307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b441499-fb38-42f7-b7b5-08c17bfbda38 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.852843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b441499-fb38-42f7-b7b5-08c17bfbda38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.865591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b441499-fb38-42f7-b7b5-08c17bfbda38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6b441499-fb38-42f7-b7b5-08c17bfbda38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:02:58.874865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:58.875830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:02:58.876657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:58.880182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 384/786] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:58.881328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b441499-fb38-42f7-b7b5-08c17bfbda38 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:58.892635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:58.895474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23d108-f339-4947-aef2-76e1609d6d38 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:58.932886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf8fa7-9bbe-4f58-aa21-d40ca4c3379b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:58.933709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8fa7-9bbe-4f58-aa21-d40ca4c3379b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:58.934102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8fa7-9bbe-4f58-aa21-d40ca4c3379b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:58.934529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8fa7-9bbe-4f58-aa21-d40ca4c3379b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:58.935426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf8fa7-9bbe-4f58-aa21-d40ca4c3379b None admin] RBAC: Adding query filter params (interface=public, service_id=0e3bfe7b88a040bc8677c6da9ac9709e, region_id=RegionOne) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:58.935892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf8fa7-9bbe-4f58-aa21-d40ca4c3379b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:58.937332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf8fa7-9bbe-4f58-aa21-d40ca4c3379b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:58.943215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 403/787] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 12:02:58 2022] GET /identity/v3/endpoints?service_id=0e3bfe7b88a040bc8677c6da9ac9709e&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:58.944255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf8fa7-9bbe-4f58-aa21-d40ca4c3379b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.003062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b441499-fb38-42f7-b7b5-08c17bfbda38 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.006262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:59.050671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b64403-4ebc-4d59-b341-73bb2c0203b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.051615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b64403-4ebc-4d59-b341-73bb2c0203b6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.052064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b64403-4ebc-4d59-b341-73bb2c0203b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.052495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b64403-4ebc-4d59-b341-73bb2c0203b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.052611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf8fa7-9bbe-4f58-aa21-d40ca4c3379b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.053471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b64403-4ebc-4d59-b341-73bb2c0203b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:59.053944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b64403-4ebc-4d59-b341-73bb2c0203b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.055767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b64403-4ebc-4d59-b341-73bb2c0203b6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.070795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 385/788] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:59.072204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b64403-4ebc-4d59-b341-73bb2c0203b6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.093936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1387a6f-a443-4193-b1f7-66964dcdbf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.094981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1387a6f-a443-4193-b1f7-66964dcdbf15 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.095096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1387a6f-a443-4193-b1f7-66964dcdbf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.095384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1387a6f-a443-4193-b1f7-66964dcdbf15 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.096230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1387a6f-a443-4193-b1f7-66964dcdbf15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:59.096611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1387a6f-a443-4193-b1f7-66964dcdbf15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.098069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1387a6f-a443-4193-b1f7-66964dcdbf15 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.111554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 404/789] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:02:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:59.112559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1387a6f-a443-4193-b1f7-66964dcdbf15 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.142351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:59.188885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b64403-4ebc-4d59-b341-73bb2c0203b6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.220442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1387a6f-a443-4193-b1f7-66964dcdbf15 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.226686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1eefbf-9992-4a9b-8aea-be213350c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.227610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1eefbf-9992-4a9b-8aea-be213350c188 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.228131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1eefbf-9992-4a9b-8aea-be213350c188 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.228589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1eefbf-9992-4a9b-8aea-be213350c188 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.230072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1eefbf-9992-4a9b-8aea-be213350c188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.231872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1eefbf-9992-4a9b-8aea-be213350c188 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.238013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 386/790] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:59.239147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1eefbf-9992-4a9b-8aea-be213350c188 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.260006 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaddff-9059-4520-a502-09bbd59f52a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.261292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaddff-9059-4520-a502-09bbd59f52a3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.261582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaddff-9059-4520-a502-09bbd59f52a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.261859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaddff-9059-4520-a502-09bbd59f52a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.270131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaddff-9059-4520-a502-09bbd59f52a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.271663 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaddff-9059-4520-a502-09bbd59f52a3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.275510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6cbaddff-9059-4520-a502-09bbd59f52a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:59.289721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:59.290811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:59.293299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 405/791] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:59.298563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaddff-9059-4520-a502-09bbd59f52a3 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.362844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1eefbf-9992-4a9b-8aea-be213350c188 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.405311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50e330-3199-49db-b938-4eb2dd87848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.406558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50e330-3199-49db-b938-4eb2dd87848f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.407044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50e330-3199-49db-b938-4eb2dd87848f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.407514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50e330-3199-49db-b938-4eb2dd87848f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.407748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:59.420861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50e330-3199-49db-b938-4eb2dd87848f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.422652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50e330-3199-49db-b938-4eb2dd87848f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.439790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaddff-9059-4520-a502-09bbd59f52a3 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6b50e330-3199-49db-b938-4eb2dd87848f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:59.442895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:59.443795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:59.444937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:59.444937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:59.444937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:59.444937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:59.451950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 387/792] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:59.459366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50e330-3199-49db-b938-4eb2dd87848f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.493865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195b6f-445d-4714-85ff-e811031c53c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.495579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195b6f-445d-4714-85ff-e811031c53c5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.496082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195b6f-445d-4714-85ff-e811031c53c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.496587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195b6f-445d-4714-85ff-e811031c53c5 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.497708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22195b6f-445d-4714-85ff-e811031c53c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:59.498236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22195b6f-445d-4714-85ff-e811031c53c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.500134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22195b6f-445d-4714-85ff-e811031c53c5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.520905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 406/793] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:02:59.526743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195b6f-445d-4714-85ff-e811031c53c5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.578276 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:59.599152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50e330-3199-49db-b938-4eb2dd87848f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.636134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c99cc-2444-40e8-9d54-e59a02d253a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.636832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c99cc-2444-40e8-9d54-e59a02d253a2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.637124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c99cc-2444-40e8-9d54-e59a02d253a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.637422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c99cc-2444-40e8-9d54-e59a02d253a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.638159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c99cc-2444-40e8-9d54-e59a02d253a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:59.638451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c99cc-2444-40e8-9d54-e59a02d253a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.639822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c99cc-2444-40e8-9d54-e59a02d253a2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.652994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 388/794] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:59.660237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c99cc-2444-40e8-9d54-e59a02d253a2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.685141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195b6f-445d-4714-85ff-e811031c53c5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.722939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d96123-d425-4f40-a269-6bc9d9faa754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.724120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d96123-d425-4f40-a269-6bc9d9faa754 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.724558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d96123-d425-4f40-a269-6bc9d9faa754 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.725026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d96123-d425-4f40-a269-6bc9d9faa754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.736612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d96123-d425-4f40-a269-6bc9d9faa754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.738536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d96123-d425-4f40-a269-6bc9d9faa754 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.741747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f3d96123-d425-4f40-a269-6bc9d9faa754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:02:59.762319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:59.771310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:59.772207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:02:59.772207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:02:59.772207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:02:59.772207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:02:59.772207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 407/795] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:02:59.772207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d96123-d425-4f40-a269-6bc9d9faa754 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.775818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c99cc-2444-40e8-9d54-e59a02d253a2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.812466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7f41b-1855-408c-b38c-026299da44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.813634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7f41b-1855-408c-b38c-026299da44cd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.814095 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7f41b-1855-408c-b38c-026299da44cd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.814610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7f41b-1855-408c-b38c-026299da44cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.823746 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7f41b-1855-408c-b38c-026299da44cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.825582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7f41b-1855-408c-b38c-026299da44cd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.828722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-bbc7f41b-1855-408c-b38c-026299da44cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:02:59.842396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:02:59.851577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 389/796] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:02:59.852539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7f41b-1855-408c-b38c-026299da44cd None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.890544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d96123-d425-4f40-a269-6bc9d9faa754 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:02:59.930245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a57fe9-6d4c-4f0d-9038-0449fc6e1c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:02:59.931806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a57fe9-6d4c-4f0d-9038-0449fc6e1c75 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:02:59.932377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a57fe9-6d4c-4f0d-9038-0449fc6e1c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:02:59.933059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a57fe9-6d4c-4f0d-9038-0449fc6e1c75 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:02:59.934183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a57fe9-6d4c-4f0d-9038-0449fc6e1c75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:02:59.934754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a57fe9-6d4c-4f0d-9038-0449fc6e1c75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:02:59.936539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a57fe9-6d4c-4f0d-9038-0449fc6e1c75 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:02:59.951868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 408/797] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:02:59.958146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a57fe9-6d4c-4f0d-9038-0449fc6e1c75 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:02:59.965524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7f41b-1855-408c-b38c-026299da44cd None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:00.002533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd73d7-e4c2-4b09-9bac-eba5ef709bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:00.003284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd73d7-e4c2-4b09-9bac-eba5ef709bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.003848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd73d7-e4c2-4b09-9bac-eba5ef709bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.004014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd73d7-e4c2-4b09-9bac-eba5ef709bc6 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.004802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd73d7-e4c2-4b09-9bac-eba5ef709bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:00.005116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd73d7-e4c2-4b09-9bac-eba5ef709bc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:00.006879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd73d7-e4c2-4b09-9bac-eba5ef709bc6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:00.025292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:00.032086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 390/798] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:00.033521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd73d7-e4c2-4b09-9bac-eba5ef709bc6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:00.079189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a57fe9-6d4c-4f0d-9038-0449fc6e1c75 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:00.097177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:00.140720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd73d7-e4c2-4b09-9bac-eba5ef709bc6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:00.151808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d785162-5ae8-41b6-aa09-49ea0f25717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:00.153013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d785162-5ae8-41b6-aa09-49ea0f25717e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.153995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d785162-5ae8-41b6-aa09-49ea0f25717e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.154512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d785162-5ae8-41b6-aa09-49ea0f25717e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.162308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d785162-5ae8-41b6-aa09-49ea0f25717e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:00.164090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d785162-5ae8-41b6-aa09-49ea0f25717e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:00.177871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3d3c7-1c57-47d1-989d-b45801f10867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:00.178919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3d3c7-1c57-47d1-989d-b45801f10867 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.179224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3d3c7-1c57-47d1-989d-b45801f10867 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.179504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3d3c7-1c57-47d1-989d-b45801f10867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-2d785162-5ae8-41b6-aa09-49ea0f25717e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:00.181243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:00.182106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:00.183761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:00.183761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:00.183761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:00.183761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:00.183761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:00.183761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:00.183761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:00.183761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:00.186240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 409/799] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:00.191996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3d3c7-1c57-47d1-989d-b45801f10867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:00.192563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e109c-1f23-4beb-8144-82143a9cd2e9 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.193063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e109c-1f23-4beb-8144-82143a9cd2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.193457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3d3c7-1c57-47d1-989d-b45801f10867 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:00.193809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e109c-1f23-4beb-8144-82143a9cd2e9 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.195672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 410/800] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:00.196592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e109c-1f23-4beb-8144-82143a9cd2e9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-bdd3d3c7-1c57-47d1-989d-b45801f10867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:00.214070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:00.215082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:00.215971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:00.215971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:00.215971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:00.215971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:00.219991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 391/801] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:00.221365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a6ce5-47c5-49b3-bdd1-06f6d3c58a66 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.221662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a6ce5-47c5-49b3-bdd1-06f6d3c58a66 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.221940 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a6ce5-47c5-49b3-bdd1-06f6d3c58a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.240186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3d8a6ce5-47c5-49b3-bdd1-06f6d3c58a66 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:00.241673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3d8a6ce5-47c5-49b3-bdd1-06f6d3c58a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:00.311547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e109c-1f23-4beb-8144-82143a9cd2e9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:00.315587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 392/802] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:00.316573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a6ce5-47c5-49b3-bdd1-06f6d3c58a66 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:00.330029 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:00.350527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda02c3c-d517-4965-9efc-15f90e1980e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:00.351572 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda02c3c-d517-4965-9efc-15f90e1980e5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.352077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda02c3c-d517-4965-9efc-15f90e1980e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.352592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda02c3c-d517-4965-9efc-15f90e1980e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.353596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda02c3c-d517-4965-9efc-15f90e1980e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:00.354122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda02c3c-d517-4965-9efc-15f90e1980e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:00.356004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda02c3c-d517-4965-9efc-15f90e1980e5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:00.361410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:00.370919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 411/803] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:00.376791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda02c3c-d517-4965-9efc-15f90e1980e5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:00.426139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8a6ce5-47c5-49b3-bdd1-06f6d3c58a66 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:00.468701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec651a-a0d1-46d3-aa7d-2aee12e9a5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:00.469381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec651a-a0d1-46d3-aa7d-2aee12e9a5da None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.469732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec651a-a0d1-46d3-aa7d-2aee12e9a5da None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.470025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec651a-a0d1-46d3-aa7d-2aee12e9a5da None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.470773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec651a-a0d1-46d3-aa7d-2aee12e9a5da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:00.471288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec651a-a0d1-46d3-aa7d-2aee12e9a5da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:00.472667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec651a-a0d1-46d3-aa7d-2aee12e9a5da None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:00.491088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 393/804] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:00.492135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec651a-a0d1-46d3-aa7d-2aee12e9a5da None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:00.503965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda02c3c-d517-4965-9efc-15f90e1980e5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:00.541850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766cf33-e137-4dae-93d2-09642269cffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:00.543919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766cf33-e137-4dae-93d2-09642269cffb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.544422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766cf33-e137-4dae-93d2-09642269cffb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.544937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766cf33-e137-4dae-93d2-09642269cffb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.545912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766cf33-e137-4dae-93d2-09642269cffb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:00.547653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766cf33-e137-4dae-93d2-09642269cffb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-2766cf33-e137-4dae-93d2-09642269cffb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:00.562571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:00.563555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 12:03:00.564418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:00.567926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 412/805] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 19 12:03:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:00.569219 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766cf33-e137-4dae-93d2-09642269cffb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:00.598444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:00.601311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec651a-a0d1-46d3-aa7d-2aee12e9a5da None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:00.617657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:00.644054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fea53-5369-4745-bedb-3d9282399e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:00.644945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fea53-5369-4745-bedb-3d9282399e12 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.645226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fea53-5369-4745-bedb-3d9282399e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.645506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fea53-5369-4745-bedb-3d9282399e12 None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/6be447727ed44a48a6b023303289e319/roles/941f412208c04a7abd395240a0339304` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.662560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fea53-5369-4745-bedb-3d9282399e12 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=6be447727ed44a48a6b023303289e319, role_id=941f412208c04a7abd395240a0339304)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:00.664022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fea53-5369-4745-bedb-3d9282399e12 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:00.684919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766cf33-e137-4dae-93d2-09642269cffb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:00.718499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db000e49-f7c2-4c4a-a963-5af16c0c0c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:00.719286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db000e49-f7c2-4c4a-a963-5af16c0c0c2a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.719568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db000e49-f7c2-4c4a-a963-5af16c0c0c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.719839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db000e49-f7c2-4c4a-a963-5af16c0c0c2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.726648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db000e49-f7c2-4c4a-a963-5af16c0c0c2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:00.728211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db000e49-f7c2-4c4a-a963-5af16c0c0c2a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-db000e49-f7c2-4c4a-a963-5af16c0c0c2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:00.742836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:00.743910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 413/806] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:00.745440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db000e49-f7c2-4c4a-a963-5af16c0c0c2a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:00.814798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 394/807] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:03:00 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/6be447727ed44a48a6b023303289e319/roles/941f412208c04a7abd395240a0339304 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:03:00.816532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e01e21-a8af-402e-9aec-1f6d782f3196 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.816894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e01e21-a8af-402e-9aec-1f6d782f3196 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.817247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e01e21-a8af-402e-9aec-1f6d782f3196 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.818874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 395/808] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:00.834559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e01e21-a8af-402e-9aec-1f6d782f3196 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:00.868903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:00.873058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db000e49-f7c2-4c4a-a963-5af16c0c0c2a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:00.925464 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a2764-a876-4453-9fd6-53daf7d7f0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:00.926242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a2764-a876-4453-9fd6-53daf7d7f0ea None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.926623 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a2764-a876-4453-9fd6-53daf7d7f0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.926970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a2764-a876-4453-9fd6-53daf7d7f0ea None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.927760 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a2764-a876-4453-9fd6-53daf7d7f0ea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:00.928134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a2764-a876-4453-9fd6-53daf7d7f0ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:00.930339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a2764-a876-4453-9fd6-53daf7d7f0ea None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:00.939433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 414/809] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 12:03:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:00.940978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f482ffa-79fe-4a6b-a9d1-3c0f8c688b11 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:00.941362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f482ffa-79fe-4a6b-a9d1-3c0f8c688b11 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:00.941713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f482ffa-79fe-4a6b-a9d1-3c0f8c688b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:00.960623 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0f482ffa-79fe-4a6b-a9d1-3c0f8c688b11 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:00.962157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0f482ffa-79fe-4a6b-a9d1-3c0f8c688b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:00.962603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e01e21-a8af-402e-9aec-1f6d782f3196 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:01.011164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcb380-8c0a-4afa-80ab-172ca38772f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:01.012123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcb380-8c0a-4afa-80ab-172ca38772f9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:01.012475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcb380-8c0a-4afa-80ab-172ca38772f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:01.012839 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcb380-8c0a-4afa-80ab-172ca38772f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:01.013761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcb380-8c0a-4afa-80ab-172ca38772f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:01.014144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcb380-8c0a-4afa-80ab-172ca38772f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:01.015938 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcb380-8c0a-4afa-80ab-172ca38772f9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:01.031781 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 396/810] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:01.032649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcb380-8c0a-4afa-80ab-172ca38772f9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:01.048161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:01.060384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 415/811] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:01.061471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f482ffa-79fe-4a6b-a9d1-3c0f8c688b11 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:01.180241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:01.183025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcb380-8c0a-4afa-80ab-172ca38772f9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:01.188275 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f482ffa-79fe-4a6b-a9d1-3c0f8c688b11 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:01.222811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c0345-7d99-45e8-ac78-3195610e7064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:01.223952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c0345-7d99-45e8-ac78-3195610e7064 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:01.224552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c0345-7d99-45e8-ac78-3195610e7064 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:01.225029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c0345-7d99-45e8-ac78-3195610e7064 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:01.225935 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c0345-7d99-45e8-ac78-3195610e7064 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:01.226470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c0345-7d99-45e8-ac78-3195610e7064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:01.227202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7457a17-6ff5-4903-98c4-09f0107d56e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:01.228103 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7457a17-6ff5-4903-98c4-09f0107d56e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:01.228381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7457a17-6ff5-4903-98c4-09f0107d56e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:01.228654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7457a17-6ff5-4903-98c4-09f0107d56e5 None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/28ddf898ad7b4e10aaf0770249b4453d/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:01.228919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c0345-7d99-45e8-ac78-3195610e7064 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:01.231742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7457a17-6ff5-4903-98c4-09f0107d56e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=28ddf898ad7b4e10aaf0770249b4453d, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:01.234041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7457a17-6ff5-4903-98c4-09f0107d56e5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:01.237760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 397/812] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 19 12:03:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:01.238858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c0345-7d99-45e8-ac78-3195610e7064 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:01.319333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:01.354758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 416/813] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:03:01 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/28ddf898ad7b4e10aaf0770249b4453d/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:03:01.354758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7457a17-6ff5-4903-98c4-09f0107d56e5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:01.357806 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c0345-7d99-45e8-ac78-3195610e7064 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:01.433430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf880a2-02d5-4381-8e0d-f002f7eecd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:01.434489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf880a2-02d5-4381-8e0d-f002f7eecd9a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:01.434778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf880a2-02d5-4381-8e0d-f002f7eecd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:01.435105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf880a2-02d5-4381-8e0d-f002f7eecd9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:01.454590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf880a2-02d5-4381-8e0d-f002f7eecd9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3bf880a2-02d5-4381-8e0d-f002f7eecd9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:01.470136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:01.471214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:01.472653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 398/814] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:03:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:01.477548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf880a2-02d5-4381-8e0d-f002f7eecd9a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:01.616380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7457a17-6ff5-4903-98c4-09f0107d56e5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:01.649677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf880a2-02d5-4381-8e0d-f002f7eecd9a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:01.658641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501488a9-677c-499c-9ba6-5be5b9fb30a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:01.659313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501488a9-677c-499c-9ba6-5be5b9fb30a6 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:01.659601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501488a9-677c-499c-9ba6-5be5b9fb30a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:01.659879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501488a9-677c-499c-9ba6-5be5b9fb30a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:01.660625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501488a9-677c-499c-9ba6-5be5b9fb30a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:01.662822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501488a9-677c-499c-9ba6-5be5b9fb30a6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:01.696216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e3a79-f67e-4d8f-aa51-ffa5d95fa40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:01.696850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e3a79-f67e-4d8f-aa51-ffa5d95fa40a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:01.697188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e3a79-f67e-4d8f-aa51-ffa5d95fa40a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:01.697467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e3a79-f67e-4d8f-aa51-ffa5d95fa40a None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:01.698593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e3a79-f67e-4d8f-aa51-ffa5d95fa40a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:01.699149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e3a79-f67e-4d8f-aa51-ffa5d95fa40a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:01.700677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968e3a79-f67e-4d8f-aa51-ffa5d95fa40a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:01.702295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:01.711757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 399/815] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:03:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:01.718151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e3a79-f67e-4d8f-aa51-ffa5d95fa40a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:01.788428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 417/816] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 12:03:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:03:01.833782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e3a79-f67e-4d8f-aa51-ffa5d95fa40a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:01.878697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf3191-c8ee-4cda-911c-c6f3bf95412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:01.879504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf3191-c8ee-4cda-911c-c6f3bf95412d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:01.879800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf3191-c8ee-4cda-911c-c6f3bf95412d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:01.880056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf3191-c8ee-4cda-911c-c6f3bf95412d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:01.892826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf3191-c8ee-4cda-911c-c6f3bf95412d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:01.894322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf3191-c8ee-4cda-911c-c6f3bf95412d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b4cf3191-c8ee-4cda-911c-c6f3bf95412d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:01.914628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:01.915597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:01.917241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:01.917241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:01.917241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:01.917241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:01.917241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 400/817] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:01.922702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501488a9-677c-499c-9ba6-5be5b9fb30a6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:01.944526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:02.072449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501488a9-677c-499c-9ba6-5be5b9fb30a6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:02.112590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ceef0-f508-494d-8de7-84707395f448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:02.113269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ceef0-f508-494d-8de7-84707395f448 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:02.113554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ceef0-f508-494d-8de7-84707395f448 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:02.113833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ceef0-f508-494d-8de7-84707395f448 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:02.114596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ceef0-f508-494d-8de7-84707395f448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:02.114912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ceef0-f508-494d-8de7-84707395f448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:02.116292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ceef0-f508-494d-8de7-84707395f448 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:02.130126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 418/818] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:02.135704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf3191-c8ee-4cda-911c-c6f3bf95412d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:02.189683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:02.248847 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf3191-c8ee-4cda-911c-c6f3bf95412d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:02.286191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e21fd5-d508-45a0-bc35-02185de3d1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:02.286940 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e21fd5-d508-45a0-bc35-02185de3d1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:02.287224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e21fd5-d508-45a0-bc35-02185de3d1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:02.287500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e21fd5-d508-45a0-bc35-02185de3d1e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:02.296541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e21fd5-d508-45a0-bc35-02185de3d1e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:02.298007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e21fd5-d508-45a0-bc35-02185de3d1e4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c0e21fd5-d508-45a0-bc35-02185de3d1e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:02.315267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:02.316296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:02.317227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 401/819] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 12:03:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:02.324016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ceef0-f508-494d-8de7-84707395f448 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:02.353778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010b77a-1e73-4885-ad26-39d7a6007581 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:02.354691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010b77a-1e73-4885-ad26-39d7a6007581 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:02.354915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010b77a-1e73-4885-ad26-39d7a6007581 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:02.356571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 402/820] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:02.361942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efecc4f-790d-4b61-98f7-d18b1a7ee97b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:02.362505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efecc4f-790d-4b61-98f7-d18b1a7ee97b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:02.363038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efecc4f-790d-4b61-98f7-d18b1a7ee97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:02.383052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8efecc4f-790d-4b61-98f7-d18b1a7ee97b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:02.384610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8efecc4f-790d-4b61-98f7-d18b1a7ee97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:02.408670 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:02.449766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ceef0-f508-494d-8de7-84707395f448 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:02.492292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc38e8-b039-404e-ae75-8f3f1548c5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:02.493339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc38e8-b039-404e-ae75-8f3f1548c5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:02.493853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc38e8-b039-404e-ae75-8f3f1548c5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:02.494338 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc38e8-b039-404e-ae75-8f3f1548c5f1 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:02.496806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc38e8-b039-404e-ae75-8f3f1548c5f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:02.497364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc38e8-b039-404e-ae75-8f3f1548c5f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:02.499511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc38e8-b039-404e-ae75-8f3f1548c5f1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:02.520827 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 419/821] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 12:03:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:02.531453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc38e8-b039-404e-ae75-8f3f1548c5f1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:02.574537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 403/822] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:02.620321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efecc4f-790d-4b61-98f7-d18b1a7ee97b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:02.680880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:02.685561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc38e8-b039-404e-ae75-8f3f1548c5f1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:02.718826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:02.732648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669dfca-19fe-49e7-8ad2-25b45b71ee59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:02.733352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669dfca-19fe-49e7-8ad2-25b45b71ee59 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:02.733637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669dfca-19fe-49e7-8ad2-25b45b71ee59 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:02.733913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669dfca-19fe-49e7-8ad2-25b45b71ee59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:02.807964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efecc4f-790d-4b61-98f7-d18b1a7ee97b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:02.846432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba88305-63dd-4ec6-ae9b-38d4f56c3b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:02.847609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba88305-63dd-4ec6-ae9b-38d4f56c3b9a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:02.848121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba88305-63dd-4ec6-ae9b-38d4f56c3b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:02.848604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba88305-63dd-4ec6-ae9b-38d4f56c3b9a None admin] PATH_INFO: `/v3/services/network` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:02.849474 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba88305-63dd-4ec6-ae9b-38d4f56c3b9a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:02.851105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba88305-63dd-4ec6-ae9b-38d4f56c3b9a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-cba88305-63dd-4ec6-ae9b-38d4f56c3b9a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:02.863907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:02.864946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 12:03:02.865777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:02.874244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 404/823] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:02 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:02.874555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669dfca-19fe-49e7-8ad2-25b45b71ee59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:02.875996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba88305-63dd-4ec6-ae9b-38d4f56c3b9a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:02.876182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669dfca-19fe-49e7-8ad2-25b45b71ee59 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:03.039313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:03.048070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba88305-63dd-4ec6-ae9b-38d4f56c3b9a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:03.083285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 420/824] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:03:03.089319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669dfca-19fe-49e7-8ad2-25b45b71ee59 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:03.118070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65ce86-cc11-4675-aaaf-f46d79b537d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:03.120700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65ce86-cc11-4675-aaaf-f46d79b537d9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:03.122063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65ce86-cc11-4675-aaaf-f46d79b537d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:03.122609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65ce86-cc11-4675-aaaf-f46d79b537d9 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:03.123715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65ce86-cc11-4675-aaaf-f46d79b537d9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:03.125629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65ce86-cc11-4675-aaaf-f46d79b537d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:03.127272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65ce86-cc11-4675-aaaf-f46d79b537d9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:03.143657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 405/825] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:03:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:03.146434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65ce86-cc11-4675-aaaf-f46d79b537d9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:03.154050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:03.300254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669dfca-19fe-49e7-8ad2-25b45b71ee59 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:03.335226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65ce86-cc11-4675-aaaf-f46d79b537d9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:03.347248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcb593-123c-4eac-a8fc-7fe3df3002f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:03.348058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcb593-123c-4eac-a8fc-7fe3df3002f6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:03.348766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcb593-123c-4eac-a8fc-7fe3df3002f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:03.349050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcb593-123c-4eac-a8fc-7fe3df3002f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:03.363840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcb593-123c-4eac-a8fc-7fe3df3002f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:03.366039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcb593-123c-4eac-a8fc-7fe3df3002f6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:03.376057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff898b38-85dd-40c1-adfa-aea17fa97535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:03.377178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff898b38-85dd-40c1-adfa-aea17fa97535 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:03.377465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff898b38-85dd-40c1-adfa-aea17fa97535 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:03.377747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff898b38-85dd-40c1-adfa-aea17fa97535 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:03.379021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff898b38-85dd-40c1-adfa-aea17fa97535 None admin] RBAC: Adding query filter params (type=network) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:03.379481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff898b38-85dd-40c1-adfa-aea17fa97535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:03.380967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff898b38-85dd-40c1-adfa-aea17fa97535 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:03.386591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 406/826] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:03:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:03.388004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559291ab-c29d-472f-8ac9-2e6e6250714f None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:03.388292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559291ab-c29d-472f-8ac9-2e6e6250714f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:03.388564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559291ab-c29d-472f-8ac9-2e6e6250714f None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:03.389860 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 407/827] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:03.392494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559291ab-c29d-472f-8ac9-2e6e6250714f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d3dcb593-123c-4eac-a8fc-7fe3df3002f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:03.396874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:03.397752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:03.398641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:03.398641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:03.398641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:03.398641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:03.398641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 421/828] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:03.403755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3f0b8-ba6a-45d0-a9d9-24f324852ed4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:03.404042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3f0b8-ba6a-45d0-a9d9-24f324852ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:03.404324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3f0b8-ba6a-45d0-a9d9-24f324852ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:03.423392 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-03d3f0b8-ba6a-45d0-a9d9-24f324852ed4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:03.424642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-03d3f0b8-ba6a-45d0-a9d9-24f324852ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:03.425685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:03.497083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 422/829] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:03.497829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3f0b8-ba6a-45d0-a9d9-24f324852ed4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:03.512868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559291ab-c29d-472f-8ac9-2e6e6250714f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:03.523443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:03.552245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70379df-bc31-4a04-985f-52ba912aa4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:03.552928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70379df-bc31-4a04-985f-52ba912aa4cc None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:03.553216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70379df-bc31-4a04-985f-52ba912aa4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:03.553518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70379df-bc31-4a04-985f-52ba912aa4cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:03.554303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70379df-bc31-4a04-985f-52ba912aa4cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:03.555743 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70379df-bc31-4a04-985f-52ba912aa4cc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:03.630585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3f0b8-ba6a-45d0-a9d9-24f324852ed4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:03.666099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 408/830] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 12:03:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:03:03.670983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70379df-bc31-4a04-985f-52ba912aa4cc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:03.702089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bd938-31e6-482b-bda2-7a2c377233ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:03.702815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bd938-31e6-482b-bda2-7a2c377233ef None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:03.703102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bd938-31e6-482b-bda2-7a2c377233ef None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:03.703383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bd938-31e6-482b-bda2-7a2c377233ef None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:03.704125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bd938-31e6-482b-bda2-7a2c377233ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:03.704440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bd938-31e6-482b-bda2-7a2c377233ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:03.705872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bd938-31e6-482b-bda2-7a2c377233ef None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:03.733574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 423/831] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:03.737805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bd938-31e6-482b-bda2-7a2c377233ef None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:03.849609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:03.867674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:03.893820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70379df-bc31-4a04-985f-52ba912aa4cc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:03.929654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bd938-31e6-482b-bda2-7a2c377233ef None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:03.944972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc8015-2c55-4aac-839f-b0d30be5246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:03.946836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc8015-2c55-4aac-839f-b0d30be5246d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:03.947337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc8015-2c55-4aac-839f-b0d30be5246d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:03.948374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc8015-2c55-4aac-839f-b0d30be5246d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:03.962872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc8015-2c55-4aac-839f-b0d30be5246d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:03.974740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac1d34-e790-4144-a5b3-0c4d63e223ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-a7cc8015-2c55-4aac-839f-b0d30be5246d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:03.975065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:03.976052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:03:03.976868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:03.977541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac1d34-e790-4144-a5b3-0c4d63e223ec None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:03.977541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac1d34-e790-4144-a5b3-0c4d63e223ec None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:03.977541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac1d34-e790-4144-a5b3-0c4d63e223ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:03.980472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 409/832] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:03:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:03.986343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc8015-2c55-4aac-839f-b0d30be5246d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:03.986675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac1d34-e790-4144-a5b3-0c4d63e223ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:03.988341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac1d34-e790-4144-a5b3-0c4d63e223ec None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-32ac1d34-e790-4144-a5b3-0c4d63e223ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:04.005534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:04.006523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:04.007406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:04.007406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:04.007406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:04.007406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:04.007406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:04.007406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:04.007406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:04.007406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:04.007406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 424/833] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:04.013421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac1d34-e790-4144-a5b3-0c4d63e223ec None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:04.115226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc8015-2c55-4aac-839f-b0d30be5246d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:04.153608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac1d34-e790-4144-a5b3-0c4d63e223ec None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:04.158715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d42b8-1779-407c-bda7-5697adc07db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:04.159405 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d42b8-1779-407c-bda7-5697adc07db3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.159686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d42b8-1779-407c-bda7-5697adc07db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.159960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d42b8-1779-407c-bda7-5697adc07db3 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.160708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d42b8-1779-407c-bda7-5697adc07db3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:04.161022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d42b8-1779-407c-bda7-5697adc07db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:04.163005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d42b8-1779-407c-bda7-5697adc07db3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:04.176173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 410/834] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:03:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:04.177523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4d17-c1ea-4981-bc8a-895a89203fa6 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.177818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4d17-c1ea-4981-bc8a-895a89203fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.178100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4d17-c1ea-4981-bc8a-895a89203fa6 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.179464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 411/835] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:04.181582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e4d17-c1ea-4981-bc8a-895a89203fa6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:04.187591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:04.187928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:04.197031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a0e9d-f089-4107-86dc-980e8d24b731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:04.197759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a0e9d-f089-4107-86dc-980e8d24b731 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.205297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a0e9d-f089-4107-86dc-980e8d24b731 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.205297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a0e9d-f089-4107-86dc-980e8d24b731 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.205297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a0e9d-f089-4107-86dc-980e8d24b731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:04.205297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a0e9d-f089-4107-86dc-980e8d24b731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:04.205297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a0e9d-f089-4107-86dc-980e8d24b731 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:04.213584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 425/836] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:04.214512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f9aa6-bcd6-4ade-a4d4-276c4cae05e3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.214791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f9aa6-bcd6-4ade-a4d4-276c4cae05e3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.215070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f9aa6-bcd6-4ade-a4d4-276c4cae05e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.234075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d00f9aa6-bcd6-4ade-a4d4-276c4cae05e3 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:04.235291 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d00f9aa6-bcd6-4ade-a4d4-276c4cae05e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:04.299648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e4d17-c1ea-4981-bc8a-895a89203fa6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:04.313654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 426/837] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:04.315521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f9aa6-bcd6-4ade-a4d4-276c4cae05e3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:04.339521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0e435-b87a-42e7-b134-b6c4bb9c217f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:04.340777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0e435-b87a-42e7-b134-b6c4bb9c217f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.341060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0e435-b87a-42e7-b134-b6c4bb9c217f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.341399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0e435-b87a-42e7-b134-b6c4bb9c217f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.353189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0e435-b87a-42e7-b134-b6c4bb9c217f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:04.355214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0e435-b87a-42e7-b134-b6c4bb9c217f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-55c0e435-b87a-42e7-b134-b6c4bb9c217f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:04.373587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:04.374603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:04.376037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:04.376037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:04.376037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:04.376037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:04.376037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 412/838] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:04.380275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0e435-b87a-42e7-b134-b6c4bb9c217f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:04.430896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f9aa6-bcd6-4ade-a4d4-276c4cae05e3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:04.443090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:04.449344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:04.467677 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e17d2-2220-48c9-9f6f-4c3e32102022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:04.468343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e17d2-2220-48c9-9f6f-4c3e32102022 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.468630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e17d2-2220-48c9-9f6f-4c3e32102022 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.468909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e17d2-2220-48c9-9f6f-4c3e32102022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.469746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e17d2-2220-48c9-9f6f-4c3e32102022 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=bb6776311f354b748d563c4f1d793972) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:04.470064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e17d2-2220-48c9-9f6f-4c3e32102022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:04.471536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e17d2-2220-48c9-9f6f-4c3e32102022 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:04.477763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 427/839] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:04 2022] GET /identity/v3/role_assignments?user.id=bb6776311f354b748d563c4f1d793972&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=bd295cb5d1194b469beff0381c6b23af => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:04.478674 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e17d2-2220-48c9-9f6f-4c3e32102022 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:04.534929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0e435-b87a-42e7-b134-b6c4bb9c217f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:04.589193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc14a-8bce-4a74-a353-c8c7009db19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:04.591199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc14a-8bce-4a74-a353-c8c7009db19b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.591658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc14a-8bce-4a74-a353-c8c7009db19b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.592094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc14a-8bce-4a74-a353-c8c7009db19b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.610438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbc14a-8bce-4a74-a353-c8c7009db19b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:04.611549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e17d2-2220-48c9-9f6f-4c3e32102022 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-aefbc14a-8bce-4a74-a353-c8c7009db19b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:04.620303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:04.621502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:04.622581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:04.623978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 413/840] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:03:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:04.638168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc14a-8bce-4a74-a353-c8c7009db19b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:04.664218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0708c4c-76da-4947-9312-770a63e550ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:04.665251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0708c4c-76da-4947-9312-770a63e550ad None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.665840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0708c4c-76da-4947-9312-770a63e550ad None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.666343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0708c4c-76da-4947-9312-770a63e550ad None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.667960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0708c4c-76da-4947-9312-770a63e550ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:04.668541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0708c4c-76da-4947-9312-770a63e550ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:04.670480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0708c4c-76da-4947-9312-770a63e550ad None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:04.686202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 428/841] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:04.691752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0708c4c-76da-4947-9312-770a63e550ad None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:04.741367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:04.764301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc14a-8bce-4a74-a353-c8c7009db19b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:04.782027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:04.804472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dafbce-e7b9-419c-9c2c-f13a9aec19e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:04.805148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dafbce-e7b9-419c-9c2c-f13a9aec19e3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.805429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dafbce-e7b9-419c-9c2c-f13a9aec19e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.805709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dafbce-e7b9-419c-9c2c-f13a9aec19e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.806500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dafbce-e7b9-419c-9c2c-f13a9aec19e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:04.806961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dafbce-e7b9-419c-9c2c-f13a9aec19e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:04.809360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dafbce-e7b9-419c-9c2c-f13a9aec19e3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:04.819182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 414/842] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:03:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:04.822381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0708c4c-76da-4947-9312-770a63e550ad None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:04.825701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012a3f-f6a9-4577-9d4e-d3ee4846f7ab None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.826389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012a3f-f6a9-4577-9d4e-d3ee4846f7ab None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.828187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32012a3f-f6a9-4577-9d4e-d3ee4846f7ab None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.832252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 415/843] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:04 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:04.833383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32012a3f-f6a9-4577-9d4e-d3ee4846f7ab None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:04.875015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570c412-2c19-4502-97ce-d5fa42d285bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:04.876150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570c412-2c19-4502-97ce-d5fa42d285bf None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.876432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570c412-2c19-4502-97ce-d5fa42d285bf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.876710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570c412-2c19-4502-97ce-d5fa42d285bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.885887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570c412-2c19-4502-97ce-d5fa42d285bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:04.887386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570c412-2c19-4502-97ce-d5fa42d285bf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6570c412-2c19-4502-97ce-d5fa42d285bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:04.903605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:04.904615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:04.905553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 429/844] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:03:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:04.906846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfa785-11d2-42f8-83e7-0efebf42da1b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.906846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfa785-11d2-42f8-83e7-0efebf42da1b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.906846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfa785-11d2-42f8-83e7-0efebf42da1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:04.926076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1ddfa785-11d2-42f8-83e7-0efebf42da1b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:04.927325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1ddfa785-11d2-42f8-83e7-0efebf42da1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:04.952806 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32012a3f-f6a9-4577-9d4e-d3ee4846f7ab None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:04.988017 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eed12d-6589-4e49-ae6f-266e908c702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:04.988962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eed12d-6589-4e49-ae6f-266e908c702c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:04.989347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eed12d-6589-4e49-ae6f-266e908c702c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:04.989728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eed12d-6589-4e49-ae6f-266e908c702c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.001118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eed12d-6589-4e49-ae6f-266e908c702c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.002811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eed12d-6589-4e49-ae6f-266e908c702c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.006515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 430/845] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:05.006515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfa785-11d2-42f8-83e7-0efebf42da1b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c0eed12d-6589-4e49-ae6f-266e908c702c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:05.025826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:05.027291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:05.028150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:05.028150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:05.028150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:05.028150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:05.029865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 416/846] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:05.030846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eed12d-6589-4e49-ae6f-266e908c702c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.046796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:05.065999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:05.125555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfa785-11d2-42f8-83e7-0efebf42da1b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:05.145754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eed12d-6589-4e49-ae6f-266e908c702c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:05.162836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284675-df86-4adf-a2c4-1173310cfc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:05.163510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284675-df86-4adf-a2c4-1173310cfc5f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:05.163794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284675-df86-4adf-a2c4-1173310cfc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:05.164069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284675-df86-4adf-a2c4-1173310cfc5f None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.164904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94284675-df86-4adf-a2c4-1173310cfc5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:05.165215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94284675-df86-4adf-a2c4-1173310cfc5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.167785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94284675-df86-4adf-a2c4-1173310cfc5f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.185706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 431/847] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:05.186747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284675-df86-4adf-a2c4-1173310cfc5f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.194913 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dab7f6-da9e-442f-b750-fced10db22fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:05.195845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dab7f6-da9e-442f-b750-fced10db22fb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:05.196031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dab7f6-da9e-442f-b750-fced10db22fb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:05.196327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dab7f6-da9e-442f-b750-fced10db22fb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.196944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dab7f6-da9e-442f-b750-fced10db22fb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.199267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dab7f6-da9e-442f-b750-fced10db22fb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-50dab7f6-da9e-442f-b750-fced10db22fb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:05.212681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:05.214288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:05.215277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 417/848] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:05.216663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dab7f6-da9e-442f-b750-fced10db22fb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.290652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:05.302956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284675-df86-4adf-a2c4-1173310cfc5f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:05.321296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:05.328852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dab7f6-da9e-442f-b750-fced10db22fb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:05.341528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab27b0-fe4e-41f9-9fc0-b0d14a995595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:05.342192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab27b0-fe4e-41f9-9fc0-b0d14a995595 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:05.342508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab27b0-fe4e-41f9-9fc0-b0d14a995595 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:05.343023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab27b0-fe4e-41f9-9fc0-b0d14a995595 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.343495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab27b0-fe4e-41f9-9fc0-b0d14a995595 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:05.343800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab27b0-fe4e-41f9-9fc0-b0d14a995595 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.345176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab27b0-fe4e-41f9-9fc0-b0d14a995595 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.362165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 432/849] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:05.362165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab27b0-fe4e-41f9-9fc0-b0d14a995595 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.367300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb9024-a6c1-4d16-bd19-ab87d5192a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:05.367958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb9024-a6c1-4d16-bd19-ab87d5192a05 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:05.368245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb9024-a6c1-4d16-bd19-ab87d5192a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:05.368523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb9024-a6c1-4d16-bd19-ab87d5192a05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.445008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb9024-a6c1-4d16-bd19-ab87d5192a05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.446858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb9024-a6c1-4d16-bd19-ab87d5192a05 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.469054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab27b0-fe4e-41f9-9fc0-b0d14a995595 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:05.504708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e3779-9d4a-4411-9d18-885726916246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:05.505365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e3779-9d4a-4411-9d18-885726916246 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:05.505646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e3779-9d4a-4411-9d18-885726916246 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:05.505920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e3779-9d4a-4411-9d18-885726916246 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.506647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e3779-9d4a-4411-9d18-885726916246 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:05.506954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e3779-9d4a-4411-9d18-885726916246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.508314 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e3779-9d4a-4411-9d18-885726916246 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.518094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 433/850] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:05.518094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e3779-9d4a-4411-9d18-885726916246 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.534396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:05.552559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:05.568047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 418/851] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:03:05.568047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb9024-a6c1-4d16-bd19-ab87d5192a05 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.630868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e3779-9d4a-4411-9d18-885726916246 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:05.667478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15881b5f-fc92-4991-af43-f045d56e33af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:05.668258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15881b5f-fc92-4991-af43-f045d56e33af None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:05.668543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15881b5f-fc92-4991-af43-f045d56e33af None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:05.668822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15881b5f-fc92-4991-af43-f045d56e33af None admin] PATH_INFO: `/v3/users/swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.676273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15881b5f-fc92-4991-af43-f045d56e33af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.677715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15881b5f-fc92-4991-af43-f045d56e33af None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.678799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb9024-a6c1-4d16-bd19-ab87d5192a05 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-15881b5f-fc92-4991-af43-f045d56e33af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:05.694366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:05.695468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 434/852] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:05.696382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15881b5f-fc92-4991-af43-f045d56e33af None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.727012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5c7fd-e822-4678-8ff6-bd0021153a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:05.727602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5c7fd-e822-4678-8ff6-bd0021153a29 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:05.727883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5c7fd-e822-4678-8ff6-bd0021153a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:05.728163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5c7fd-e822-4678-8ff6-bd0021153a29 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.729497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5c7fd-e822-4678-8ff6-bd0021153a29 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:05.729830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5c7fd-e822-4678-8ff6-bd0021153a29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.731234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5c7fd-e822-4678-8ff6-bd0021153a29 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.735844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 419/853] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:05.743827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5c7fd-e822-4678-8ff6-bd0021153a29 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.794100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:05.829815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15881b5f-fc92-4991-af43-f045d56e33af None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:05.834877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:05.873651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9571959-3824-4c70-bae1-c3aae4368b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:05.874917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9571959-3824-4c70-bae1-c3aae4368b2b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:05.875461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9571959-3824-4c70-bae1-c3aae4368b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:05.875949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9571959-3824-4c70-bae1-c3aae4368b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.890050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9571959-3824-4c70-bae1-c3aae4368b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.891956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9571959-3824-4c70-bae1-c3aae4368b2b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.892086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5c7fd-e822-4678-8ff6-bd0021153a29 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-c9571959-3824-4c70-bae1-c3aae4368b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:05.913049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:05.914038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:05.914934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:05.914934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:05.914934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:05.914934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:05.917915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 435/854] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:05.923720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9571959-3824-4c70-bae1-c3aae4368b2b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:05.934848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f259535-1831-4ece-b7b0-a650cbe27d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:05.935533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f259535-1831-4ece-b7b0-a650cbe27d44 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:05.935818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f259535-1831-4ece-b7b0-a650cbe27d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:05.936091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f259535-1831-4ece-b7b0-a650cbe27d44 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:05.936928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f259535-1831-4ece-b7b0-a650cbe27d44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:05.937624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f259535-1831-4ece-b7b0-a650cbe27d44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:05.939535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f259535-1831-4ece-b7b0-a650cbe27d44 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:05.962196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 420/855] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:05.968045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f259535-1831-4ece-b7b0-a650cbe27d44 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:06.053384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9571959-3824-4c70-bae1-c3aae4368b2b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:06.070611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f259535-1831-4ece-b7b0-a650cbe27d44 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:06.080250 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:06.092350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2a891-3510-41ae-ae85-5576a7a16f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:06.093117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2a891-3510-41ae-ae85-5576a7a16f6b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.093399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2a891-3510-41ae-ae85-5576a7a16f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.093727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2a891-3510-41ae-ae85-5576a7a16f6b None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.094590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2a891-3510-41ae-ae85-5576a7a16f6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:06.094896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2a891-3510-41ae-ae85-5576a7a16f6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:06.096274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2a891-3510-41ae-ae85-5576a7a16f6b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:06.102772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:06.107383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1760d-d762-487b-9879-ade9776071f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:06.108041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1760d-d762-487b-9879-ade9776071f9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.108324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1760d-d762-487b-9879-ade9776071f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.108600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1760d-d762-487b-9879-ade9776071f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.110105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 436/856] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:06.115359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2a891-3510-41ae-ae85-5576a7a16f6b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:06.182943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1760d-d762-487b-9879-ade9776071f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:06.184408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1760d-d762-487b-9879-ade9776071f9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:06.243237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2a891-3510-41ae-ae85-5576a7a16f6b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:06.289676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 421/857] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 12:03:06.292416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06a77-3664-4ca3-8e6c-a05933b2bd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:06.293230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06a77-3664-4ca3-8e6c-a05933b2bd49 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.293514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06a77-3664-4ca3-8e6c-a05933b2bd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.293793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06a77-3664-4ca3-8e6c-a05933b2bd49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.295562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1760d-d762-487b-9879-ade9776071f9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:06.308657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c06a77-3664-4ca3-8e6c-a05933b2bd49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:06.310978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c06a77-3664-4ca3-8e6c-a05933b2bd49 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:06.315426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-79c06a77-3664-4ca3-8e6c-a05933b2bd49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:06.332494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:06.333626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:06.334761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:06.334761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:06.334761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:06.334761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:06.334761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:06.334761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:06.334761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:06.334761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:06.337879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 437/858] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:06.344642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06a77-3664-4ca3-8e6c-a05933b2bd49 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:06.393471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:06.405982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1760d-d762-487b-9879-ade9776071f9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:06.442987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d2a68-5cfd-4104-a13f-74e52f761093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:06.443787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d2a68-5cfd-4104-a13f-74e52f761093 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.444084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d2a68-5cfd-4104-a13f-74e52f761093 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.444484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d2a68-5cfd-4104-a13f-74e52f761093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.453160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06a77-3664-4ca3-8e6c-a05933b2bd49 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:06.456607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d2a68-5cfd-4104-a13f-74e52f761093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:06.457859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d2a68-5cfd-4104-a13f-74e52f761093 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-318d2a68-5cfd-4104-a13f-74e52f761093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:06.481642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:06.482648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:06.484090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:06.484090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:06.484090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:06.484090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:06.484090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 422/859] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:06.488313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d2a68-5cfd-4104-a13f-74e52f761093 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:06.502500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c39416-5a95-41b6-9cdc-f4170957ee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:06.503972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c39416-5a95-41b6-9cdc-f4170957ee40 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.504469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c39416-5a95-41b6-9cdc-f4170957ee40 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.505382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c39416-5a95-41b6-9cdc-f4170957ee40 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.506736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c39416-5a95-41b6-9cdc-f4170957ee40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:06.507279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c39416-5a95-41b6-9cdc-f4170957ee40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:06.509706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c39416-5a95-41b6-9cdc-f4170957ee40 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:06.533136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 438/860] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:06.534349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c39416-5a95-41b6-9cdc-f4170957ee40 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:06.577937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:06.612915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d2a68-5cfd-4104-a13f-74e52f761093 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:06.655258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3205fb-67ed-4731-927e-1a9aa2bfcaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:06.655950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3205fb-67ed-4731-927e-1a9aa2bfcaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.656236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3205fb-67ed-4731-927e-1a9aa2bfcaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.656511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3205fb-67ed-4731-927e-1a9aa2bfcaf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.657234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3205fb-67ed-4731-927e-1a9aa2bfcaf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:06.657543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3205fb-67ed-4731-927e-1a9aa2bfcaf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:06.659177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3205fb-67ed-4731-927e-1a9aa2bfcaf4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:06.676604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 423/861] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:06.678100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3205fb-67ed-4731-927e-1a9aa2bfcaf4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:06.684766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c39416-5a95-41b6-9cdc-f4170957ee40 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:06.688170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:06.736494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd6473-415e-4ffe-b118-15a2ae58df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:06.737873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd6473-415e-4ffe-b118-15a2ae58df50 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.738344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd6473-415e-4ffe-b118-15a2ae58df50 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.738825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd6473-415e-4ffe-b118-15a2ae58df50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.739872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd6473-415e-4ffe-b118-15a2ae58df50 None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=6be447727ed44a48a6b023303289e319) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:06.740876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd6473-415e-4ffe-b118-15a2ae58df50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:06.743063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd6473-415e-4ffe-b118-15a2ae58df50 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:06.759023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 439/862] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:06 2022] GET /identity/v3/role_assignments?user.id=6be447727ed44a48a6b023303289e319&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:06.805835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3205fb-67ed-4731-927e-1a9aa2bfcaf4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:06.842297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:06.851832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ba523-c1ca-4e9c-b36c-056009782673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:06.853117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ba523-c1ca-4e9c-b36c-056009782673 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.853599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ba523-c1ca-4e9c-b36c-056009782673 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.854614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ba523-c1ca-4e9c-b36c-056009782673 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.863142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ba523-c1ca-4e9c-b36c-056009782673 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:06.864933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ba523-c1ca-4e9c-b36c-056009782673 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-566ba523-c1ca-4e9c-b36c-056009782673 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:06.882829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:06.883829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:06.884782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:06.884782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:06.884782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:06.884782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:06.884782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:06.884782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:06.884782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:06.884782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:06.888268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 424/863] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:06.894557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd6473-415e-4ffe-b118-15a2ae58df50 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:06.918123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4f4ef-6731-4f20-9021-f9a6b8d5de24 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.918510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4f4ef-6731-4f20-9021-f9a6b8d5de24 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.918737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4f4ef-6731-4f20-9021-f9a6b8d5de24 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.920508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 425/864] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:06.926566 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15c055-ed04-45ed-b084-bf9923e48b53 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:06.926902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15c055-ed04-45ed-b084-bf9923e48b53 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:06.927147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15c055-ed04-45ed-b084-bf9923e48b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:06.950976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7e15c055-ed04-45ed-b084-bf9923e48b53 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:06.951417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7e15c055-ed04-45ed-b084-bf9923e48b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:07.033651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 426/865] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:07.041932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd6473-415e-4ffe-b118-15a2ae58df50 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:07.055612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15c055-ed04-45ed-b084-bf9923e48b53 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:07.085359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aba995-59b1-4c19-8c36-d162277e28be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:07.086038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aba995-59b1-4c19-8c36-d162277e28be None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:07.086321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aba995-59b1-4c19-8c36-d162277e28be None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:07.086621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aba995-59b1-4c19-8c36-d162277e28be None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:07.087412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aba995-59b1-4c19-8c36-d162277e28be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:07.087726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aba995-59b1-4c19-8c36-d162277e28be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:07.089974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aba995-59b1-4c19-8c36-d162277e28be None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:07.103633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 440/866] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:07.112517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aba995-59b1-4c19-8c36-d162277e28be None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:07.149359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:07.178673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15c055-ed04-45ed-b084-bf9923e48b53 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:07.206080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:07.225511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1438b-a180-471b-9c82-6436e52e31d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:07.226456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1438b-a180-471b-9c82-6436e52e31d4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:07.226745 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1438b-a180-471b-9c82-6436e52e31d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:07.227023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1438b-a180-471b-9c82-6436e52e31d4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:07.227785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1438b-a180-471b-9c82-6436e52e31d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:07.229625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1438b-a180-471b-9c82-6436e52e31d4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-45b1438b-a180-471b-9c82-6436e52e31d4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:07.245141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:07.247285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:07.250601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 427/867] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 12:03:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:07.253976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1438b-a180-471b-9c82-6436e52e31d4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:07.254904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aba995-59b1-4c19-8c36-d162277e28be None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:07.294468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9e66d-b07e-4084-8e82-0e52399fb272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:07.295138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9e66d-b07e-4084-8e82-0e52399fb272 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:07.295451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9e66d-b07e-4084-8e82-0e52399fb272 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:07.295675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9e66d-b07e-4084-8e82-0e52399fb272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:07.296815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9e66d-b07e-4084-8e82-0e52399fb272 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=28ddf898ad7b4e10aaf0770249b4453d) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:07.296925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9e66d-b07e-4084-8e82-0e52399fb272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:07.298353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9e66d-b07e-4084-8e82-0e52399fb272 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:07.304852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 441/868] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:07 2022] GET /identity/v3/role_assignments?user.id=28ddf898ad7b4e10aaf0770249b4453d&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=bd295cb5d1194b469beff0381c6b23af => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:07.381435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1438b-a180-471b-9c82-6436e52e31d4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:07.421930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ba639-efbd-49eb-a5d0-787ee7e8e637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:07.422630 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ba639-efbd-49eb-a5d0-787ee7e8e637 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:07.422910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ba639-efbd-49eb-a5d0-787ee7e8e637 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:07.423307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ba639-efbd-49eb-a5d0-787ee7e8e637 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:07.424024 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ba639-efbd-49eb-a5d0-787ee7e8e637 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:07.424334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ba639-efbd-49eb-a5d0-787ee7e8e637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:07.425715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ba639-efbd-49eb-a5d0-787ee7e8e637 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:07.431270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 428/869] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:07.437295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9e66d-b07e-4084-8e82-0e52399fb272 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:07.547797 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:07.558811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9e66d-b07e-4084-8e82-0e52399fb272 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:07.600648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9392a4-a60c-40bd-b7dd-4b916add7019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:07.601380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9392a4-a60c-40bd-b7dd-4b916add7019 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:07.601670 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9392a4-a60c-40bd-b7dd-4b916add7019 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:07.601948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9392a4-a60c-40bd-b7dd-4b916add7019 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:07.602849 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9392a4-a60c-40bd-b7dd-4b916add7019 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:07.603154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9392a4-a60c-40bd-b7dd-4b916add7019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:07.605123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9392a4-a60c-40bd-b7dd-4b916add7019 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:07.610281 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 442/870] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:08.245707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36c291-adbc-46be-a2b5-66b14299fb66 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:08.246103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36c291-adbc-46be-a2b5-66b14299fb66 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:08.246283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36c291-adbc-46be-a2b5-66b14299fb66 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:08.247799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 429/871] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:08.253345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0ed76-d9ac-4fc4-8042-c9881e111e31 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:08.253641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0ed76-d9ac-4fc4-8042-c9881e111e31 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:08.253919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0ed76-d9ac-4fc4-8042-c9881e111e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:08.272094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-bbe0ed76-d9ac-4fc4-8042-c9881e111e31 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:08.273359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bbe0ed76-d9ac-4fc4-8042-c9881e111e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:08.346798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 443/872] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:08.369706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36c291-adbc-46be-a2b5-66b14299fb66 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:08.419598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:08.479228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36c291-adbc-46be-a2b5-66b14299fb66 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:08.515909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee868b-9b0b-4223-996d-47d35e3364aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:08.516778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee868b-9b0b-4223-996d-47d35e3364aa None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:08.517057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee868b-9b0b-4223-996d-47d35e3364aa None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:08.517329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee868b-9b0b-4223-996d-47d35e3364aa None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:08.518192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee868b-9b0b-4223-996d-47d35e3364aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:08.519590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee868b-9b0b-4223-996d-47d35e3364aa None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:08.601455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 430/873] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:03:08 2022] POST /identity/v3/services => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:03:09.383019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666a60b-b834-4585-a49c-b58539e7b8f5 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:09.383892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666a60b-b834-4585-a49c-b58539e7b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:09.384327 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666a60b-b834-4585-a49c-b58539e7b8f5 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:09.386715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 444/874] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:09.392601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bfec3-f41c-4b7e-be02-3fc6dedbf666 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:09.393173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bfec3-f41c-4b7e-be02-3fc6dedbf666 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:09.393665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bfec3-f41c-4b7e-be02-3fc6dedbf666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:09.411210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-613bfec3-f41c-4b7e-be02-3fc6dedbf666 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:09.412744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-613bfec3-f41c-4b7e-be02-3fc6dedbf666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:09.497028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 431/875] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:09.521023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666a60b-b834-4585-a49c-b58539e7b8f5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:09.618648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:09.636240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666a60b-b834-4585-a49c-b58539e7b8f5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:09.677211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e85e3f-8234-4ba1-a90a-9aeac226b877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:09.678052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e85e3f-8234-4ba1-a90a-9aeac226b877 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:09.678334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e85e3f-8234-4ba1-a90a-9aeac226b877 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:09.678645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e85e3f-8234-4ba1-a90a-9aeac226b877 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:09.692033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e85e3f-8234-4ba1-a90a-9aeac226b877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-d9e85e3f-8234-4ba1-a90a-9aeac226b877 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:09.701713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:09.702776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:09.703613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 445/876] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 12:03:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:09.710865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bfec3-f41c-4b7e-be02-3fc6dedbf666 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:09.743794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:09.825820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bfec3-f41c-4b7e-be02-3fc6dedbf666 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:09.868362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d3f2c-33cf-42a1-a242-f0b4d59ce90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:09.870342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d3f2c-33cf-42a1-a242-f0b4d59ce90e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:09.870857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d3f2c-33cf-42a1-a242-f0b4d59ce90e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:09.871317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d3f2c-33cf-42a1-a242-f0b4d59ce90e None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:09.872088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d3f2c-33cf-42a1-a242-f0b4d59ce90e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:09.872389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d3f2c-33cf-42a1-a242-f0b4d59ce90e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:09.875751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b0a58-0b50-43e3-ac9c-bab524edb73a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:09.875888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d3f2c-33cf-42a1-a242-f0b4d59ce90e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:09.876066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b0a58-0b50-43e3-ac9c-bab524edb73a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:09.876305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b0a58-0b50-43e3-ac9c-bab524edb73a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:09.878198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 446/877] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:09.885206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 432/878] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 12:03:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:09.887454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406adef-b1de-44fd-b435-088d18832dee None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:09.887679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406adef-b1de-44fd-b435-088d18832dee None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:09.887953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406adef-b1de-44fd-b435-088d18832dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:09.891446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d3f2c-33cf-42a1-a242-f0b4d59ce90e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:09.906692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f406adef-b1de-44fd-b435-088d18832dee None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:09.907887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f406adef-b1de-44fd-b435-088d18832dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:09.984639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 447/879] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:09.986311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c440d-51f2-4b33-98fb-0586464e0ce3 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:09.986771 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c440d-51f2-4b33-98fb-0586464e0ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:09.987236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c440d-51f2-4b33-98fb-0586464e0ce3 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:09.988904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 448/880] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:09.999683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6fe03-ee07-4164-bce1-9244970314a7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.000193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6fe03-ee07-4164-bce1-9244970314a7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.000624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6fe03-ee07-4164-bce1-9244970314a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.004421 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:10.011671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d3f2c-33cf-42a1-a242-f0b4d59ce90e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:10.022107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-90d6fe03-ee07-4164-bce1-9244970314a7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:10.023940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-90d6fe03-ee07-4164-bce1-9244970314a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:10.048867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85bbd-1f2b-4ed8-8ce6-399aa4275403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:10.049945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85bbd-1f2b-4ed8-8ce6-399aa4275403 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.050279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85bbd-1f2b-4ed8-8ce6-399aa4275403 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.050641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc85bbd-1f2b-4ed8-8ce6-399aa4275403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.063092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc85bbd-1f2b-4ed8-8ce6-399aa4275403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:10.064543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc85bbd-1f2b-4ed8-8ce6-399aa4275403 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-edc85bbd-1f2b-4ed8-8ce6-399aa4275403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:10.084930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:10.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:10.086787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:10.086787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:10.086787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:10.086787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:10.086787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 433/881] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:10.087768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85bbd-1f2b-4ed8-8ce6-399aa4275403 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:10.105057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 449/882] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:10.105916 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6fe03-ee07-4164-bce1-9244970314a7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:10.114310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:10.205324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc85bbd-1f2b-4ed8-8ce6-399aa4275403 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:10.212364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6fe03-ee07-4164-bce1-9244970314a7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:10.241705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab237bc-dde4-44c2-ab94-3ce758cfc998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:10.242541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab237bc-dde4-44c2-ab94-3ce758cfc998 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.242829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab237bc-dde4-44c2-ab94-3ce758cfc998 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.243100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab237bc-dde4-44c2-ab94-3ce758cfc998 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.243749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab237bc-dde4-44c2-ab94-3ce758cfc998 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:10.245739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab237bc-dde4-44c2-ab94-3ce758cfc998 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:10.249957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b4624-c6b1-4f93-822a-2e5d71e4b56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:10.250815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b4624-c6b1-4f93-822a-2e5d71e4b56e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.251107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b4624-c6b1-4f93-822a-2e5d71e4b56e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.251387 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b4624-c6b1-4f93-822a-2e5d71e4b56e None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.252119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b4624-c6b1-4f93-822a-2e5d71e4b56e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:10.252431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b4624-c6b1-4f93-822a-2e5d71e4b56e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:10.253790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b4624-c6b1-4f93-822a-2e5d71e4b56e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-eab237bc-dde4-44c2-ab94-3ce758cfc998 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:10.258200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:10.259309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:10.266130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 434/883] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:10.266943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab237bc-dde4-44c2-ab94-3ce758cfc998 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:10.267012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 450/884] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:10.267889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b4624-c6b1-4f93-822a-2e5d71e4b56e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:10.279174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:10.339465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:10.374331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab237bc-dde4-44c2-ab94-3ce758cfc998 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:10.375492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b4624-c6b1-4f93-822a-2e5d71e4b56e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:10.410203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a4fe-5dbf-4126-af9c-3f77805beaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:10.410974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a4fe-5dbf-4126-af9c-3f77805beaef None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.411240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a4fe-5dbf-4126-af9c-3f77805beaef None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.411583 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a4fe-5dbf-4126-af9c-3f77805beaef None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.412214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1a4fe-5dbf-4126-af9c-3f77805beaef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:10.412524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1a4fe-5dbf-4126-af9c-3f77805beaef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:10.412747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5773216-146c-4894-aa34-7ac4f88a1dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:10.413411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5773216-146c-4894-aa34-7ac4f88a1dd0 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.413697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5773216-146c-4894-aa34-7ac4f88a1dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.413889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1a4fe-5dbf-4126-af9c-3f77805beaef None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:10.413986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5773216-146c-4894-aa34-7ac4f88a1dd0 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.414747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5773216-146c-4894-aa34-7ac4f88a1dd0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:10.416923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5773216-146c-4894-aa34-7ac4f88a1dd0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:10.424576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 451/885] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:10.424576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a4fe-5dbf-4126-af9c-3f77805beaef None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:10.523753 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a4fe-5dbf-4126-af9c-3f77805beaef None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:10.527173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 435/886] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:03:10 2022] POST /identity/v3/services => generated 269 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:03:10.527804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5773216-146c-4894-aa34-7ac4f88a1dd0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:10.565815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:10.580426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bf0ee-aad1-4250-96e6-279997ac9dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:10.581528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bf0ee-aad1-4250-96e6-279997ac9dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.582105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bf0ee-aad1-4250-96e6-279997ac9dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.582571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bf0ee-aad1-4250-96e6-279997ac9dc7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.592996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bf0ee-aad1-4250-96e6-279997ac9dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:10.594922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bf0ee-aad1-4250-96e6-279997ac9dc7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f72bf0ee-aad1-4250-96e6-279997ac9dc7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:10.613026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:10.614264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:03:10.615532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:10.623723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:10.624625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 452/887] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:03:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:10.625733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bf0ee-aad1-4250-96e6-279997ac9dc7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:10.659814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5773216-146c-4894-aa34-7ac4f88a1dd0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:10.700348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a335e0-e962-4242-9110-913b3353ac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:10.701378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a335e0-e962-4242-9110-913b3353ac72 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.701770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a335e0-e962-4242-9110-913b3353ac72 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.702518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a335e0-e962-4242-9110-913b3353ac72 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.703044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a335e0-e962-4242-9110-913b3353ac72 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:10.703456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a335e0-e962-4242-9110-913b3353ac72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:10.705063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a335e0-e962-4242-9110-913b3353ac72 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:10.716316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 436/888] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:10.762570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bf0ee-aad1-4250-96e6-279997ac9dc7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:10.802865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3413d1d-9ccf-462a-97ff-6a0839b4679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:10.803705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3413d1d-9ccf-462a-97ff-6a0839b4679b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.804214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3413d1d-9ccf-462a-97ff-6a0839b4679b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.804497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3413d1d-9ccf-462a-97ff-6a0839b4679b None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.805360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3413d1d-9ccf-462a-97ff-6a0839b4679b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:10.805670 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3413d1d-9ccf-462a-97ff-6a0839b4679b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:10.808055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3413d1d-9ccf-462a-97ff-6a0839b4679b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:10.828871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 453/889] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 12:03:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:10.834593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a335e0-e962-4242-9110-913b3353ac72 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:10.902528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7947478-373d-405c-9053-1894d4f1c4f7 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.903020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7947478-373d-405c-9053-1894d4f1c4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.903302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7947478-373d-405c-9053-1894d4f1c4f7 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.905476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 454/890] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:10.911400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4ce00-f447-4514-8b90-ac8d892763ca None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:10.911681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4ce00-f447-4514-8b90-ac8d892763ca None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:10.911961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4ce00-f447-4514-8b90-ac8d892763ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:10.934562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1cc4ce00-f447-4514-8b90-ac8d892763ca None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:10.935765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1cc4ce00-f447-4514-8b90-ac8d892763ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:10.967334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a335e0-e962-4242-9110-913b3353ac72 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:10.984440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:11.014762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 455/891] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:11.023407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb96c-98e1-4259-9d5a-c52ad002bdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:11.023407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb96c-98e1-4259-9d5a-c52ad002bdbf None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:11.023407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb96c-98e1-4259-9d5a-c52ad002bdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:11.023407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb96c-98e1-4259-9d5a-c52ad002bdbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:11.032843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:11.048145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4ce00-f447-4514-8b90-ac8d892763ca None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:11.112511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb96c-98e1-4259-9d5a-c52ad002bdbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:11.114338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb96c-98e1-4259-9d5a-c52ad002bdbf None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:11.160597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4ce00-f447-4514-8b90-ac8d892763ca None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:11.199974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c42e9b-a3b8-4631-843f-91f6f415ab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:11.200774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c42e9b-a3b8-4631-843f-91f6f415ab20 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:11.201057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c42e9b-a3b8-4631-843f-91f6f415ab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:11.201330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c42e9b-a3b8-4631-843f-91f6f415ab20 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:11.202203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c42e9b-a3b8-4631-843f-91f6f415ab20 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:11.203840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c42e9b-a3b8-4631-843f-91f6f415ab20 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-50c42e9b-a3b8-4631-843f-91f6f415ab20 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:11.216877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:11.217957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 12:03:11.218868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:11.223249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 456/892] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:03:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:11.224202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c42e9b-a3b8-4631-843f-91f6f415ab20 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:11.237086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 437/893] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:10 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:03:11.238338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb96c-98e1-4259-9d5a-c52ad002bdbf None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:11.268608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:11.274489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:11.335023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c42e9b-a3b8-4631-843f-91f6f415ab20 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:11.345856 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb96c-98e1-4259-9d5a-c52ad002bdbf None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:11.374756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8de9a-4952-4a31-9d64-623306955c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:11.375424 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8de9a-4952-4a31-9d64-623306955c16 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:11.375626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8de9a-4952-4a31-9d64-623306955c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:11.375906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8de9a-4952-4a31-9d64-623306955c16 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:11.376608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8de9a-4952-4a31-9d64-623306955c16 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:11.376917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8de9a-4952-4a31-9d64-623306955c16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:11.378381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8de9a-4952-4a31-9d64-623306955c16 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:11.387379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068bfa-5fcb-4cab-8b12-f12fef011b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:11.388484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068bfa-5fcb-4cab-8b12-f12fef011b91 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:11.388979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068bfa-5fcb-4cab-8b12-f12fef011b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:11.389232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 457/894] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:03:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:11.389692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068bfa-5fcb-4cab-8b12-f12fef011b91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:11.390124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8de9a-4952-4a31-9d64-623306955c16 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:11.401766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59068bfa-5fcb-4cab-8b12-f12fef011b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:11.403623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59068bfa-5fcb-4cab-8b12-f12fef011b91 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-59068bfa-5fcb-4cab-8b12-f12fef011b91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:11.422993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:11.423943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:11.424838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:11.424838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:11.424838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:11.424838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:11.434701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 438/895] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:11.436768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068bfa-5fcb-4cab-8b12-f12fef011b91 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:11.500333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8de9a-4952-4a31-9d64-623306955c16 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:11.501987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:11.533382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:11.539065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe91e25-f0ce-4559-b868-9577ec1d91cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:11.539778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe91e25-f0ce-4559-b868-9577ec1d91cf None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:11.540059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe91e25-f0ce-4559-b868-9577ec1d91cf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:11.540334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe91e25-f0ce-4559-b868-9577ec1d91cf None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:11.542446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe91e25-f0ce-4559-b868-9577ec1d91cf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:11.543335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe91e25-f0ce-4559-b868-9577ec1d91cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:11.549429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe91e25-f0ce-4559-b868-9577ec1d91cf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:11.554122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068bfa-5fcb-4cab-8b12-f12fef011b91 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:11.555213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 458/896] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:03:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:11.558581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe91e25-f0ce-4559-b868-9577ec1d91cf None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:11.597286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759f1e9-db87-46a8-bd84-41a7f4470e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:11.598397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759f1e9-db87-46a8-bd84-41a7f4470e5d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:11.598927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759f1e9-db87-46a8-bd84-41a7f4470e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:11.599412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759f1e9-db87-46a8-bd84-41a7f4470e5d None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:11.600454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759f1e9-db87-46a8-bd84-41a7f4470e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:11.600972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759f1e9-db87-46a8-bd84-41a7f4470e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:11.602683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759f1e9-db87-46a8-bd84-41a7f4470e5d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:11.621914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 439/897] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:11.623291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759f1e9-db87-46a8-bd84-41a7f4470e5d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:11.672426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe91e25-f0ce-4559-b868-9577ec1d91cf None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:11.711376 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba67095-5a71-4f88-9493-1df7261b0eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:11.711962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67095-5a71-4f88-9493-1df7261b0eeb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:11.712252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67095-5a71-4f88-9493-1df7261b0eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:11.712525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba67095-5a71-4f88-9493-1df7261b0eeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:11.714535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba67095-5a71-4f88-9493-1df7261b0eeb None admin] RBAC: Adding query filter params (interface=public, service_id=11182e6b7acb40f6a9db940b3495e72a, region_id=RegionOne) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:11.714854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba67095-5a71-4f88-9493-1df7261b0eeb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:11.716279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba67095-5a71-4f88-9493-1df7261b0eeb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:11.726593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 459/898] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 12:03:11 2022] GET /identity/v3/endpoints?service_id=11182e6b7acb40f6a9db940b3495e72a&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:11.727546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba67095-5a71-4f88-9493-1df7261b0eeb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:11.740234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759f1e9-db87-46a8-bd84-41a7f4470e5d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:11.764385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:11.780838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37c932-704d-48d2-b995-65a36ac47c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:11.782047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37c932-704d-48d2-b995-65a36ac47c8b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:11.782603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37c932-704d-48d2-b995-65a36ac47c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:11.783113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37c932-704d-48d2-b995-65a36ac47c8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:11.790633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37c932-704d-48d2-b995-65a36ac47c8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:11.792997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37c932-704d-48d2-b995-65a36ac47c8b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4a37c932-704d-48d2-b995-65a36ac47c8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:11.812066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:11.813046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:11.813919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:11.813919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:11.813919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:11.813919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:11.813919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:11.813919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:11.813919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:11.813919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:11.820755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 440/899] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:11.821452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37c932-704d-48d2-b995-65a36ac47c8b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:11.834072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:11.852402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba67095-5a71-4f88-9493-1df7261b0eeb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:11.894848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4efba-411b-4a01-96c3-f14e1cb957ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:11.895540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4efba-411b-4a01-96c3-f14e1cb957ea None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:11.895830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4efba-411b-4a01-96c3-f14e1cb957ea None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:11.896283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4efba-411b-4a01-96c3-f14e1cb957ea None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:11.897411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4efba-411b-4a01-96c3-f14e1cb957ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:11.899285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4efba-411b-4a01-96c3-f14e1cb957ea None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:11.909071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 460/900] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 12:03:11 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:03:11.961602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37c932-704d-48d2-b995-65a36ac47c8b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:12.002208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f0a37-b8ea-4aff-88b7-62732a0222d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:12.002923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f0a37-b8ea-4aff-88b7-62732a0222d6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:12.003213 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f0a37-b8ea-4aff-88b7-62732a0222d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:12.003489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f0a37-b8ea-4aff-88b7-62732a0222d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:12.004307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f0a37-b8ea-4aff-88b7-62732a0222d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:12.004621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f0a37-b8ea-4aff-88b7-62732a0222d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:12.006206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f0a37-b8ea-4aff-88b7-62732a0222d6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:12.023862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 441/901] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:12.032483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4efba-411b-4a01-96c3-f14e1cb957ea None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:12.139689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:12.147155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4efba-411b-4a01-96c3-f14e1cb957ea None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:12.183400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4a6b7-d69e-4c20-bcab-5d95807e8d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:12.184070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4a6b7-d69e-4c20-bcab-5d95807e8d37 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:12.184356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4a6b7-d69e-4c20-bcab-5d95807e8d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:12.184681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4a6b7-d69e-4c20-bcab-5d95807e8d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:12.185531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4a6b7-d69e-4c20-bcab-5d95807e8d37 None admin] RBAC: Adding query filter params (role.id=d538a3ae543f4c1897c8c3d9c8fb8066, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=6be447727ed44a48a6b023303289e319) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:12.185848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4a6b7-d69e-4c20-bcab-5d95807e8d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:12.187361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4a6b7-d69e-4c20-bcab-5d95807e8d37 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:12.194191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 461/902] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:12 2022] GET /identity/v3/role_assignments?user.id=6be447727ed44a48a6b023303289e319&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=d538a3ae543f4c1897c8c3d9c8fb8066 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:12.903383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa9d54-cd8b-4f8c-b5fe-86e10325024f None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:12.903932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa9d54-cd8b-4f8c-b5fe-86e10325024f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:12.903932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa9d54-cd8b-4f8c-b5fe-86e10325024f None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:12.905883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 442/903] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:12.911377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70ab2-48e3-423e-9fb2-9a8876e8e9cf None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:12.911801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70ab2-48e3-423e-9fb2-9a8876e8e9cf None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:12.911956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70ab2-48e3-423e-9fb2-9a8876e8e9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:12.929803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9fd70ab2-48e3-423e-9fb2-9a8876e8e9cf None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:12.931015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9fd70ab2-48e3-423e-9fb2-9a8876e8e9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:13.010694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 462/904] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:13.026619 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa9d54-cd8b-4f8c-b5fe-86e10325024f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:13.037684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6222ef8-aced-43a4-9288-663ecf91d38b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:13.037984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6222ef8-aced-43a4-9288-663ecf91d38b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:13.038267 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6222ef8-aced-43a4-9288-663ecf91d38b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:13.039913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 463/905] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:13.046203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b89d5-90a0-43f9-8608-82fce6ba7f03 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:13.046519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b89d5-90a0-43f9-8608-82fce6ba7f03 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:13.046800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b89d5-90a0-43f9-8608-82fce6ba7f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:13.068270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-319b89d5-90a0-43f9-8608-82fce6ba7f03 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:13.069694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-319b89d5-90a0-43f9-8608-82fce6ba7f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:13.116845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:13.166981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 464/906] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:13.169091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa9d54-cd8b-4f8c-b5fe-86e10325024f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:13.183272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b89d5-90a0-43f9-8608-82fce6ba7f03 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:13.202687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:13.214284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04b92-8dc8-44fe-b3be-9583a863652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:13.215151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04b92-8dc8-44fe-b3be-9583a863652b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:13.215591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04b92-8dc8-44fe-b3be-9583a863652b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:13.215874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04b92-8dc8-44fe-b3be-9583a863652b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:13.216860 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c04b92-8dc8-44fe-b3be-9583a863652b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:13.218521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c04b92-8dc8-44fe-b3be-9583a863652b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-11c04b92-8dc8-44fe-b3be-9583a863652b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:13.235159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:13.236131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 12:03:13.236971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:13.242559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 443/907] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 12:03:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:13.244975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04b92-8dc8-44fe-b3be-9583a863652b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:13.315633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b89d5-90a0-43f9-8608-82fce6ba7f03 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:13.357430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959a486-ff8f-43f9-af72-520ada2a8271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:13.358187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959a486-ff8f-43f9-af72-520ada2a8271 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:13.358594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959a486-ff8f-43f9-af72-520ada2a8271 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:13.358939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959a486-ff8f-43f9-af72-520ada2a8271 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:13.359747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959a486-ff8f-43f9-af72-520ada2a8271 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:13.361167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04b92-8dc8-44fe-b3be-9583a863652b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:13.361320 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959a486-ff8f-43f9-af72-520ada2a8271 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:13.394982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:13.400891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34ca34-61ac-4f5a-b34d-b937402d9bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:13.401588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34ca34-61ac-4f5a-b34d-b937402d9bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:13.401868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34ca34-61ac-4f5a-b34d-b937402d9bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:13.402142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34ca34-61ac-4f5a-b34d-b937402d9bb8 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:13.402908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34ca34-61ac-4f5a-b34d-b937402d9bb8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:13.403203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34ca34-61ac-4f5a-b34d-b937402d9bb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:13.404563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34ca34-61ac-4f5a-b34d-b937402d9bb8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:13.409413 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 444/908] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:13.414978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34ca34-61ac-4f5a-b34d-b937402d9bb8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:13.490507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 465/909] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:03:13 2022] POST /identity/v3/services => generated 241 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:03:13.537663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34ca34-61ac-4f5a-b34d-b937402d9bb8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:13.584074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7e00-d820-457b-a7f7-9703c5a21c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:13.584751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7e00-d820-457b-a7f7-9703c5a21c8d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:13.585086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7e00-d820-457b-a7f7-9703c5a21c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:13.585370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7e00-d820-457b-a7f7-9703c5a21c8d None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:13.586079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b7e00-d820-457b-a7f7-9703c5a21c8d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:13.586389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b7e00-d820-457b-a7f7-9703c5a21c8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:13.587846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b7e00-d820-457b-a7f7-9703c5a21c8d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:13.598515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 445/910] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:13.600946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959a486-ff8f-43f9-af72-520ada2a8271 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:13.695103 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:13.718700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959a486-ff8f-43f9-af72-520ada2a8271 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:13.754726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3695d-ca06-47f9-8a2a-6268b9cb49aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:13.755744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3695d-ca06-47f9-8a2a-6268b9cb49aa None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:13.756251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3695d-ca06-47f9-8a2a-6268b9cb49aa None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:13.756732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3695d-ca06-47f9-8a2a-6268b9cb49aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:13.757809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3695d-ca06-47f9-8a2a-6268b9cb49aa None admin] RBAC: Adding query filter params (interface=public, service_id=92aba8529cd84d37a58f2ed3bc57033f, region_id=RegionOne) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:13.758342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3695d-ca06-47f9-8a2a-6268b9cb49aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:13.760480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3695d-ca06-47f9-8a2a-6268b9cb49aa None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:13.766581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 466/911] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 12:03:13 2022] GET /identity/v3/endpoints?service_id=92aba8529cd84d37a58f2ed3bc57033f&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:13.772121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7e00-d820-457b-a7f7-9703c5a21c8d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:13.833870 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:13.883999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7e00-d820-457b-a7f7-9703c5a21c8d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:13.923114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639650f7-e383-486c-a1ad-fa893225bb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:13.924479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639650f7-e383-486c-a1ad-fa893225bb5e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:13.924912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639650f7-e383-486c-a1ad-fa893225bb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:13.925335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639650f7-e383-486c-a1ad-fa893225bb5e None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:13.926194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639650f7-e383-486c-a1ad-fa893225bb5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:13.927786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639650f7-e383-486c-a1ad-fa893225bb5e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:13.942440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 446/912] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 12:03:13 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:03:14.331648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9097bd1-9bf7-4463-8db3-ddea02b8cf8b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:14.332177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9097bd1-9bf7-4463-8db3-ddea02b8cf8b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:14.332177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9097bd1-9bf7-4463-8db3-ddea02b8cf8b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:14.333826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 467/913] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:14.340859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef49e5-2587-4582-b1eb-7f156d433361 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:14.341172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef49e5-2587-4582-b1eb-7f156d433361 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:14.341416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef49e5-2587-4582-b1eb-7f156d433361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:14.360459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8cef49e5-2587-4582-b1eb-7f156d433361 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:14.361671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8cef49e5-2587-4582-b1eb-7f156d433361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:14.437392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 447/914] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:14.460876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9097bd1-9bf7-4463-8db3-ddea02b8cf8b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:14.546331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d20f96-e85b-40bf-bdd3-f30777d62fe5 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:14.553717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d20f96-e85b-40bf-bdd3-f30777d62fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:14.553717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d20f96-e85b-40bf-bdd3-f30777d62fe5 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:14.553717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 448/915] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:14.554767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599fa44-e050-4e23-99e9-189f69e1047c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:14.555263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599fa44-e050-4e23-99e9-189f69e1047c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:14.555764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599fa44-e050-4e23-99e9-189f69e1047c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:14.572628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9097bd1-9bf7-4463-8db3-ddea02b8cf8b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:14.574294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a599fa44-e050-4e23-99e9-189f69e1047c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:14.575735 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a599fa44-e050-4e23-99e9-189f69e1047c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:14.610059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4a923-dbb1-480e-a37b-52367b33f65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:14.611151 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4a923-dbb1-480e-a37b-52367b33f65b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:14.611438 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4a923-dbb1-480e-a37b-52367b33f65b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:14.611718 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4a923-dbb1-480e-a37b-52367b33f65b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:14.612437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4a923-dbb1-480e-a37b-52367b33f65b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:14.614298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4a923-dbb1-480e-a37b-52367b33f65b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-2ee4a923-dbb1-480e-a37b-52367b33f65b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:14.631294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:14.632373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:14.633256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 468/916] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:03:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:14.636258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:14.643711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4a923-dbb1-480e-a37b-52367b33f65b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:14.659482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:14.665524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 449/917] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:14.696394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599fa44-e050-4e23-99e9-189f69e1047c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:14.776860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4a923-dbb1-480e-a37b-52367b33f65b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:14.821523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5f242-db0f-4aec-af0b-e56f78e2e01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:14.822497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5f242-db0f-4aec-af0b-e56f78e2e01e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:14.822780 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5f242-db0f-4aec-af0b-e56f78e2e01e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:14.823077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5f242-db0f-4aec-af0b-e56f78e2e01e None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:14.824319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5f242-db0f-4aec-af0b-e56f78e2e01e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:14.824633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5f242-db0f-4aec-af0b-e56f78e2e01e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:14.826304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5f242-db0f-4aec-af0b-e56f78e2e01e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:14.827400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599fa44-e050-4e23-99e9-189f69e1047c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:14.834448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 469/918] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:03:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:14.842994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5f242-db0f-4aec-af0b-e56f78e2e01e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:14.872132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099af55c-0579-403a-b6e3-6b7039f031e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:14.872863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099af55c-0579-403a-b6e3-6b7039f031e4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:14.873144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099af55c-0579-403a-b6e3-6b7039f031e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:14.873463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099af55c-0579-403a-b6e3-6b7039f031e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:14.886401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099af55c-0579-403a-b6e3-6b7039f031e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-099af55c-0579-403a-b6e3-6b7039f031e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:14.895652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:14.896589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:14.897443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:14.902736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 450/919] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 12:03:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:14.903670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099af55c-0579-403a-b6e3-6b7039f031e4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:14.944240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:14.947195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:14.986949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5f242-db0f-4aec-af0b-e56f78e2e01e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:15.036575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635c58e-443a-4a41-b434-1e14476d375e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:15.037331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099af55c-0579-403a-b6e3-6b7039f031e4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:15.037826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635c58e-443a-4a41-b434-1e14476d375e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:15.038336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635c58e-443a-4a41-b434-1e14476d375e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:15.039061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635c58e-443a-4a41-b434-1e14476d375e None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:15.039983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635c58e-443a-4a41-b434-1e14476d375e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:15.040499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635c58e-443a-4a41-b434-1e14476d375e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:15.042984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635c58e-443a-4a41-b434-1e14476d375e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:15.049895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 470/920] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 19 12:03:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:15.059079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635c58e-443a-4a41-b434-1e14476d375e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:15.116212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae97e7a-c1c9-4c57-9341-36d3e0f3f229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:15.116897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae97e7a-c1c9-4c57-9341-36d3e0f3f229 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:15.117192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae97e7a-c1c9-4c57-9341-36d3e0f3f229 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:15.117477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae97e7a-c1c9-4c57-9341-36d3e0f3f229 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:15.118254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae97e7a-c1c9-4c57-9341-36d3e0f3f229 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:15.118599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae97e7a-c1c9-4c57-9341-36d3e0f3f229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:15.121434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae97e7a-c1c9-4c57-9341-36d3e0f3f229 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:15.138994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 451/921] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 12:03:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:15.143440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae97e7a-c1c9-4c57-9341-36d3e0f3f229 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:15.215568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635c58e-443a-4a41-b434-1e14476d375e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:15.289838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08d944-8fd4-466a-ab87-88163a648123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:15.292580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08d944-8fd4-466a-ab87-88163a648123 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:15.293057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08d944-8fd4-466a-ab87-88163a648123 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:15.294157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08d944-8fd4-466a-ab87-88163a648123 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:15.295426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08d944-8fd4-466a-ab87-88163a648123 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:15.297985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08d944-8fd4-466a-ab87-88163a648123 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:15.307770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:15.341421 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:15.345031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae97e7a-c1c9-4c57-9341-36d3e0f3f229 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:15.386586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d2abb-295d-4b94-bb84-7cf5d24814e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:15.387448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d2abb-295d-4b94-bb84-7cf5d24814e8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:15.387739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d2abb-295d-4b94-bb84-7cf5d24814e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:15.388022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d2abb-295d-4b94-bb84-7cf5d24814e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:15.400062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d2abb-295d-4b94-bb84-7cf5d24814e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:15.401420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d2abb-295d-4b94-bb84-7cf5d24814e8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-045d2abb-295d-4b94-bb84-7cf5d24814e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:15.419708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:15.420649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:15.421664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:15.421664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:15.421664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:15.421664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:15.427517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 452/922] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:15.427517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d2abb-295d-4b94-bb84-7cf5d24814e8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:15.444048 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 471/923] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 12:03:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:03:15.562889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d2abb-295d-4b94-bb84-7cf5d24814e8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:15.608149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92b7a7-9dba-426f-a09a-35846184a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:15.608828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92b7a7-9dba-426f-a09a-35846184a382 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:15.609168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92b7a7-9dba-426f-a09a-35846184a382 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:15.609455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92b7a7-9dba-426f-a09a-35846184a382 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:15.610191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92b7a7-9dba-426f-a09a-35846184a382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:15.610541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92b7a7-9dba-426f-a09a-35846184a382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:15.611945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92b7a7-9dba-426f-a09a-35846184a382 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:15.626731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 453/924] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:15.632473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08d944-8fd4-466a-ab87-88163a648123 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:15.755029 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08d944-8fd4-466a-ab87-88163a648123 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:15.792904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d1dc5-5eb5-49f9-8379-f8829b9c7dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:15.793729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1dc5-5eb5-49f9-8379-f8829b9c7dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:15.794020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1dc5-5eb5-49f9-8379-f8829b9c7dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:15.794302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d1dc5-5eb5-49f9-8379-f8829b9c7dc0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:15.803378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d1dc5-5eb5-49f9-8379-f8829b9c7dc0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:15.804852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d1dc5-5eb5-49f9-8379-f8829b9c7dc0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f29d1dc5-5eb5-49f9-8379-f8829b9c7dc0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:15.820814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:15.821825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:15.823345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 472/925] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:03:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:15.828805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92b7a7-9dba-426f-a09a-35846184a382 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:15.841352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:15.907683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa101e-9de9-4f4e-a736-ef67ef3fcab4 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:15.908010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa101e-9de9-4f4e-a736-ef67ef3fcab4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:15.908229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa101e-9de9-4f4e-a736-ef67ef3fcab4 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:15.909880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 473/926] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:15.915002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab99c5-ec4f-4028-9814-f0a22787ac3f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:15.915291 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab99c5-ec4f-4028-9814-f0a22787ac3f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:15.915567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab99c5-ec4f-4028-9814-f0a22787ac3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:15.935290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-68ab99c5-ec4f-4028-9814-f0a22787ac3f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:15.936478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-68ab99c5-ec4f-4028-9814-f0a22787ac3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:15.940960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92b7a7-9dba-426f-a09a-35846184a382 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:15.985777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:15.987868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1a10d-4c6b-4a40-8d1e-79cda881ea66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:15.988616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1a10d-4c6b-4a40-8d1e-79cda881ea66 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:15.989112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1a10d-4c6b-4a40-8d1e-79cda881ea66 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:15.989446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1a10d-4c6b-4a40-8d1e-79cda881ea66 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:15.990653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1a10d-4c6b-4a40-8d1e-79cda881ea66 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:15.990973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1a10d-4c6b-4a40-8d1e-79cda881ea66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:15.992419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1a10d-4c6b-4a40-8d1e-79cda881ea66 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.014864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 474/927] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:16.022214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 454/928] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 12:03:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:16.026538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab99c5-ec4f-4028-9814-f0a22787ac3f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.040195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1a10d-4c6b-4a40-8d1e-79cda881ea66 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.128112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:16.152869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab99c5-ec4f-4028-9814-f0a22787ac3f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:16.167380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1a10d-4c6b-4a40-8d1e-79cda881ea66 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:16.201064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc373a-7595-47cc-8769-3981fbb791aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:16.201866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc373a-7595-47cc-8769-3981fbb791aa None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.202148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc373a-7595-47cc-8769-3981fbb791aa None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.202437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc373a-7595-47cc-8769-3981fbb791aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.215821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc373a-7595-47cc-8769-3981fbb791aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:16.217270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc373a-7595-47cc-8769-3981fbb791aa None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.218110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabea67-c660-4edd-b520-b9b34ea2959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:16.219477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabea67-c660-4edd-b520-b9b34ea2959b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.220349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabea67-c660-4edd-b520-b9b34ea2959b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.220826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabea67-c660-4edd-b520-b9b34ea2959b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.221681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabea67-c660-4edd-b520-b9b34ea2959b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:16.223348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabea67-c660-4edd-b520-b9b34ea2959b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-63fc373a-7595-47cc-8769-3981fbb791aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:16.236287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:16.237178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:16.238117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:16.238117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:16.238117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:16.238117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-ecabea67-c660-4edd-b520-b9b34ea2959b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:16.240658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:16.241605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:16.242468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:16.243194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 475/929] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:16.243194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc373a-7595-47cc-8769-3981fbb791aa None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.250460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 455/930] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:16.251484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabea67-c660-4edd-b520-b9b34ea2959b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.290457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:16.362470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabea67-c660-4edd-b520-b9b34ea2959b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:16.371727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc373a-7595-47cc-8769-3981fbb791aa None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:16.393785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:16.400071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8314d-9f74-47fc-b9f4-b13dabbdca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:16.400966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8314d-9f74-47fc-b9f4-b13dabbdca80 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.401355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8314d-9f74-47fc-b9f4-b13dabbdca80 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.401665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8314d-9f74-47fc-b9f4-b13dabbdca80 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.402295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8314d-9f74-47fc-b9f4-b13dabbdca80 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:16.402813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8314d-9f74-47fc-b9f4-b13dabbdca80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:16.407672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8314d-9f74-47fc-b9f4-b13dabbdca80 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.413539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 456/931] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:16.413539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fbd44-d836-427b-ba8d-7dd353282f1b None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.413539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fbd44-d836-427b-ba8d-7dd353282f1b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.413539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fbd44-d836-427b-ba8d-7dd353282f1b None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.413539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 457/932] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:16.416230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c8057-7a52-4ebd-8194-166dcba77ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:16.417204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fbd44-d836-427b-ba8d-7dd353282f1b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.417315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8057-7a52-4ebd-8194-166dcba77ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.417590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8057-7a52-4ebd-8194-166dcba77ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.417818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8057-7a52-4ebd-8194-166dcba77ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.418636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c8057-7a52-4ebd-8194-166dcba77ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:16.418961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c8057-7a52-4ebd-8194-166dcba77ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:16.420347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c8057-7a52-4ebd-8194-166dcba77ba4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.437933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 476/933] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:16.437933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8374c8-2a87-44be-9731-eba89094296e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.437933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8374c8-2a87-44be-9731-eba89094296e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.437933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8374c8-2a87-44be-9731-eba89094296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.452695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ce8374c8-2a87-44be-9731-eba89094296e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:16.453904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ce8374c8-2a87-44be-9731-eba89094296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:16.529984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 477/934] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:16.531206 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8374c8-2a87-44be-9731-eba89094296e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.539499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fbd44-d836-427b-ba8d-7dd353282f1b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:16.568769 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:16.578750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc252bb-f572-4d5a-b4e8-23f94f1d49db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:16.579397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc252bb-f572-4d5a-b4e8-23f94f1d49db None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.579680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc252bb-f572-4d5a-b4e8-23f94f1d49db None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.579957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc252bb-f572-4d5a-b4e8-23f94f1d49db None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.580670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc252bb-f572-4d5a-b4e8-23f94f1d49db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:16.580985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc252bb-f572-4d5a-b4e8-23f94f1d49db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:16.582349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc252bb-f572-4d5a-b4e8-23f94f1d49db None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.592220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 458/935] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:16.592220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc252bb-f572-4d5a-b4e8-23f94f1d49db None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.654424 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8374c8-2a87-44be-9731-eba89094296e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:16.658264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:16.692222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2bfca-5f96-40ff-9e64-92f683be86f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:16.693021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2bfca-5f96-40ff-9e64-92f683be86f9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.693300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2bfca-5f96-40ff-9e64-92f683be86f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.693567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2bfca-5f96-40ff-9e64-92f683be86f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.701325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2bfca-5f96-40ff-9e64-92f683be86f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:16.702811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2bfca-5f96-40ff-9e64-92f683be86f9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.704369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc252bb-f572-4d5a-b4e8-23f94f1d49db None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-20f2bfca-5f96-40ff-9e64-92f683be86f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:16.717596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:16.718570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:16.719463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:16.719463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:16.719463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:16.719463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:16.719463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:16.719463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:16.719463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:16.719463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:16.719463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 478/936] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:16.719944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2bfca-5f96-40ff-9e64-92f683be86f9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.749974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb350b-6db2-464c-809f-232805537304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:16.750832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb350b-6db2-464c-809f-232805537304 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.751104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb350b-6db2-464c-809f-232805537304 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.751376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb350b-6db2-464c-809f-232805537304 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.752031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb350b-6db2-464c-809f-232805537304 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:16.753445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb350b-6db2-464c-809f-232805537304 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-9efb350b-6db2-464c-809f-232805537304 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:16.766552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:16.767512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:16.768351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 459/937] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:16.769211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb350b-6db2-464c-809f-232805537304 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.829345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:16.847904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2bfca-5f96-40ff-9e64-92f683be86f9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:16.882641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cd434-9c84-4cbd-80e9-7e75c4ea34cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:16.883304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cd434-9c84-4cbd-80e9-7e75c4ea34cf None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.883596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cd434-9c84-4cbd-80e9-7e75c4ea34cf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.883887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cd434-9c84-4cbd-80e9-7e75c4ea34cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.885515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cd434-9c84-4cbd-80e9-7e75c4ea34cf None admin] RBAC: Adding query filter params (interface=public, service_id=f89669e4112d4b6f968836e5622c7a47, region_id=RegionOne) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:16.886185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cd434-9c84-4cbd-80e9-7e75c4ea34cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:16.887855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb350b-6db2-464c-809f-232805537304 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:16.888195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cd434-9c84-4cbd-80e9-7e75c4ea34cf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.898084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 479/938] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/endpoints?service_id=f89669e4112d4b6f968836e5622c7a47&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:16.898084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cd434-9c84-4cbd-80e9-7e75c4ea34cf None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:16.915208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:16.928316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae7ad-9a53-4d8d-a751-c3aadf378a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:16.929379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae7ad-9a53-4d8d-a751-c3aadf378a67 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:16.929379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae7ad-9a53-4d8d-a751-c3aadf378a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:16.929536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae7ad-9a53-4d8d-a751-c3aadf378a67 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:16.930499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae7ad-9a53-4d8d-a751-c3aadf378a67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:16.930927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae7ad-9a53-4d8d-a751-c3aadf378a67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:16.933003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae7ad-9a53-4d8d-a751-c3aadf378a67 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:16.951102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 460/939] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:16.951102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae7ad-9a53-4d8d-a751-c3aadf378a67 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:17.007738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cd434-9c84-4cbd-80e9-7e75c4ea34cf None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:17.046978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6aeb33-75d4-4bc4-8eb7-c331938a7bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:17.047636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6aeb33-75d4-4bc4-8eb7-c331938a7bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:17.047918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6aeb33-75d4-4bc4-8eb7-c331938a7bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:17.048191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6aeb33-75d4-4bc4-8eb7-c331938a7bd7 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:17.048883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6aeb33-75d4-4bc4-8eb7-c331938a7bd7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:17.049185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6aeb33-75d4-4bc4-8eb7-c331938a7bd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:17.050564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6aeb33-75d4-4bc4-8eb7-c331938a7bd7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:17.060423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 480/940] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:17.060423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6aeb33-75d4-4bc4-8eb7-c331938a7bd7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:17.063180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae7ad-9a53-4d8d-a751-c3aadf378a67 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:17.073929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:17.108602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba092248-a51a-4e0e-bddc-7ba481796bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:17.109665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba092248-a51a-4e0e-bddc-7ba481796bab None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:17.110302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba092248-a51a-4e0e-bddc-7ba481796bab None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:17.110750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba092248-a51a-4e0e-bddc-7ba481796bab None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:17.111618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba092248-a51a-4e0e-bddc-7ba481796bab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:17.113125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba092248-a51a-4e0e-bddc-7ba481796bab None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:17.120200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 461/941] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 12:03:16 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:03:17.121327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba092248-a51a-4e0e-bddc-7ba481796bab None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:17.192079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:17.202333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6aeb33-75d4-4bc4-8eb7-c331938a7bd7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:17.247653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca97a93-4f4b-4e64-9d4e-a6bfd3826403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:17.248920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca97a93-4f4b-4e64-9d4e-a6bfd3826403 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:17.249464 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca97a93-4f4b-4e64-9d4e-a6bfd3826403 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:17.249883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca97a93-4f4b-4e64-9d4e-a6bfd3826403 None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/6be447727ed44a48a6b023303289e319/roles/d538a3ae543f4c1897c8c3d9c8fb8066` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:17.252623 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca97a93-4f4b-4e64-9d4e-a6bfd3826403 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=6be447727ed44a48a6b023303289e319, role_id=d538a3ae543f4c1897c8c3d9c8fb8066)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:17.254265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca97a93-4f4b-4e64-9d4e-a6bfd3826403 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:17.255635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba092248-a51a-4e0e-bddc-7ba481796bab None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:17.296079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7ea14-24f6-4eaf-bd30-d631d10b875b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:17.296763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7ea14-24f6-4eaf-bd30-d631d10b875b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:17.297051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7ea14-24f6-4eaf-bd30-d631d10b875b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:17.297329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7ea14-24f6-4eaf-bd30-d631d10b875b None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:17.298054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7ea14-24f6-4eaf-bd30-d631d10b875b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:17.298390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7ea14-24f6-4eaf-bd30-d631d10b875b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:17.299793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7ea14-24f6-4eaf-bd30-d631d10b875b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:17.305943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 462/942] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:17.312300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7ea14-24f6-4eaf-bd30-d631d10b875b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:17.379578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 481/943] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:03:17 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/6be447727ed44a48a6b023303289e319/roles/d538a3ae543f4c1897c8c3d9c8fb8066 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:03:17.444264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7ea14-24f6-4eaf-bd30-d631d10b875b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:17.468596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:17.484814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37032f4e-9f54-4835-968f-763ee6ba3224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:17.485546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37032f4e-9f54-4835-968f-763ee6ba3224 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:17.485879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37032f4e-9f54-4835-968f-763ee6ba3224 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:17.486201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37032f4e-9f54-4835-968f-763ee6ba3224 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:17.487019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37032f4e-9f54-4835-968f-763ee6ba3224 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:17.488525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37032f4e-9f54-4835-968f-763ee6ba3224 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:17.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 463/944] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 12:03:17 2022] POST /identity/v3/endpoints => generated 347 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:03:17.919195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814969f-f7b7-4db4-90c3-90cbe819cddc None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:17.919718 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814969f-f7b7-4db4-90c3-90cbe819cddc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:17.919968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814969f-f7b7-4db4-90c3-90cbe819cddc None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:17.921470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 482/945] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:17.926970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc605f1-d20e-4d80-a3eb-abdd9bf70bfe None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:17.927258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc605f1-d20e-4d80-a3eb-abdd9bf70bfe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:17.927539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc605f1-d20e-4d80-a3eb-abdd9bf70bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:17.946239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ccc605f1-d20e-4d80-a3eb-abdd9bf70bfe None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:17.948426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ccc605f1-d20e-4d80-a3eb-abdd9bf70bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:18.030263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 464/946] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:18.054067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814969f-f7b7-4db4-90c3-90cbe819cddc None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:18.138775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:18.164743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814969f-f7b7-4db4-90c3-90cbe819cddc None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:18.201361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:18.202040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:18.202325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:18.202636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:18.203358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:18.204802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:18.206226 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WARNING keystone.server.flask.common [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 12:03:18.264639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.sql.core [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 12:03:18.264639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 12:03:18.264639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [parameters: {'id': '572f2a3b969c43458333cff8fee7fe8f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 12:03:18.264639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 572f2a3b969c43458333cff8fee7fe8f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 572f2a3b969c43458333cff8fee7fe8f. May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 12:03:18.272626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 12:03:18.280719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application yield resource May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.rollback() May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.force_reraise() May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise self.value May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application session.commit() May 19 12:03:18.281667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._prepare_impl() May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.flush() May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._flush(objects) May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application compat.raise_( May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application flush_context.execute() May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 12:03:18.283160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rec.execute(self) May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 12:03:18.284051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 12:03:18.284830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [parameters: {'id': '572f2a3b969c43458333cff8fee7fe8f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:18.285711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:18.286624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 572f2a3b969c43458333cff8fee7fe8f. May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:18.287542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 483/947] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 19 12:03:18 2022] POST /identity/v3/projects => generated 262 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:18.290443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc605f1-d20e-4d80-a3eb-abdd9bf70bfe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:18.397677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:18.407504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc605f1-d20e-4d80-a3eb-abdd9bf70bfe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:18.442878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71e9c2-991f-49bf-b06b-06050db643c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:18.443678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e9c2-991f-49bf-b06b-06050db643c0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:18.443963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e9c2-991f-49bf-b06b-06050db643c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:18.444241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e9c2-991f-49bf-b06b-06050db643c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:18.451359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71e9c2-991f-49bf-b06b-06050db643c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:18.452803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71e9c2-991f-49bf-b06b-06050db643c0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-9a71e9c2-991f-49bf-b06b-06050db643c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:18.467528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:18.468409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:18.469469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:18.469469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:18.469469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:18.469469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:18.469469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:18.469469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:18.469469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:18.469469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:18.469469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 465/948] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:18.476871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:18.593532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc3003-c130-4089-924f-69f55c44df74 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:18.621217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:18.630540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e7b6f-2dbe-478c-af14-2486f040bcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:18.631208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7b6f-2dbe-478c-af14-2486f040bcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:18.631494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7b6f-2dbe-478c-af14-2486f040bcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:18.631770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e7b6f-2dbe-478c-af14-2486f040bcb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:18.632569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e7b6f-2dbe-478c-af14-2486f040bcb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:18.632881 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e7b6f-2dbe-478c-af14-2486f040bcb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:18.635409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e7b6f-2dbe-478c-af14-2486f040bcb7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:18.657017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 484/949] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:03:18 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:19.454964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279bb0e-6821-4995-9d9f-1ca21504f605 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:19.455624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279bb0e-6821-4995-9d9f-1ca21504f605 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:19.455624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279bb0e-6821-4995-9d9f-1ca21504f605 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:19.457212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 466/950] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:19.462697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c097d8b-f15e-4ab7-ba93-7fcb9213b80b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:19.463598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c097d8b-f15e-4ab7-ba93-7fcb9213b80b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:19.464095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c097d8b-f15e-4ab7-ba93-7fcb9213b80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:19.483462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7c097d8b-f15e-4ab7-ba93-7fcb9213b80b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:19.486446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7c097d8b-f15e-4ab7-ba93-7fcb9213b80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:19.565271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 485/951] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:19.588326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279bb0e-6821-4995-9d9f-1ca21504f605 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:19.698340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279bb0e-6821-4995-9d9f-1ca21504f605 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:19.701171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f6cf9-d7f5-4340-99dd-154beaf202ea None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:19.701664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f6cf9-d7f5-4340-99dd-154beaf202ea None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:19.702097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f6cf9-d7f5-4340-99dd-154beaf202ea None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:19.705192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 486/952] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:19.709519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34869f43-52f9-4b16-874b-6a7783b7ba42 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:19.710082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34869f43-52f9-4b16-874b-6a7783b7ba42 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:19.710626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34869f43-52f9-4b16-874b-6a7783b7ba42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:19.729328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-34869f43-52f9-4b16-874b-6a7783b7ba42 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:19.730978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-34869f43-52f9-4b16-874b-6a7783b7ba42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:19.733772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf34fe4-68b7-43b2-a238-3a0534dce816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:19.735254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf34fe4-68b7-43b2-a238-3a0534dce816 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:19.735254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf34fe4-68b7-43b2-a238-3a0534dce816 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:19.735254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf34fe4-68b7-43b2-a238-3a0534dce816 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:19.735888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf34fe4-68b7-43b2-a238-3a0534dce816 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:19.737261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf34fe4-68b7-43b2-a238-3a0534dce816 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4cf34fe4-68b7-43b2-a238-3a0534dce816 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:19.749780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:19.750989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:19.752494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 467/953] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:19.757142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf34fe4-68b7-43b2-a238-3a0534dce816 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:19.777616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:19.818506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 487/954] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:19.838104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34869f43-52f9-4b16-874b-6a7783b7ba42 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:19.874542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf34fe4-68b7-43b2-a238-3a0534dce816 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:19.896543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:19.913258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f109-7d57-405e-98d2-39751115b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:19.914067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f109-7d57-405e-98d2-39751115b1eb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:19.914244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f109-7d57-405e-98d2-39751115b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:19.914503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f109-7d57-405e-98d2-39751115b1eb None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:19.915367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924f109-7d57-405e-98d2-39751115b1eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:19.915791 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924f109-7d57-405e-98d2-39751115b1eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:19.918027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924f109-7d57-405e-98d2-39751115b1eb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:19.924237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 468/955] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:19.929453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f109-7d57-405e-98d2-39751115b1eb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:19.970819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34869f43-52f9-4b16-874b-6a7783b7ba42 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.010285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b29828-d820-4475-850a-b1b2591fa036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.011786 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b29828-d820-4475-850a-b1b2591fa036 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.012287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b29828-d820-4475-850a-b1b2591fa036 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.012767 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b29828-d820-4475-850a-b1b2591fa036 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.027003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b29828-d820-4475-850a-b1b2591fa036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-c5b29828-d820-4475-850a-b1b2591fa036 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:20.036397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:20.037370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:20.038311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:20.043540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 488/956] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 12:03:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:20.045091 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b29828-d820-4475-850a-b1b2591fa036 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:20.052047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:20.082001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f109-7d57-405e-98d2-39751115b1eb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.129711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba673803-0fb1-41bb-b480-aa76585ac1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.130817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba673803-0fb1-41bb-b480-aa76585ac1af None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.131329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba673803-0fb1-41bb-b480-aa76585ac1af None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.132050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba673803-0fb1-41bb-b480-aa76585ac1af None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.133561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba673803-0fb1-41bb-b480-aa76585ac1af None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:20.134086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba673803-0fb1-41bb-b480-aa76585ac1af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:20.136654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba673803-0fb1-41bb-b480-aa76585ac1af None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:20.143099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 469/957] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:20.145196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa0633-a281-43dd-acca-e72ab3c5be14 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.146119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa0633-a281-43dd-acca-e72ab3c5be14 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.146685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa0633-a281-43dd-acca-e72ab3c5be14 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.148696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 470/958] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:20.149572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa0633-a281-43dd-acca-e72ab3c5be14 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:20.173360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b29828-d820-4475-850a-b1b2591fa036 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.183127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:20.216874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54646e-5d5b-4ac3-846f-14eb870dce2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.217559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54646e-5d5b-4ac3-846f-14eb870dce2c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.218181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54646e-5d5b-4ac3-846f-14eb870dce2c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.218490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54646e-5d5b-4ac3-846f-14eb870dce2c None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.219276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54646e-5d5b-4ac3-846f-14eb870dce2c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:20.219570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54646e-5d5b-4ac3-846f-14eb870dce2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:20.221102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54646e-5d5b-4ac3-846f-14eb870dce2c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:20.230361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 489/959] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 12:03:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:20.231160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac9075-8e16-47bc-bd3d-a0a83db89d3b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.231455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac9075-8e16-47bc-bd3d-a0a83db89d3b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.231726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac9075-8e16-47bc-bd3d-a0a83db89d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.248243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-04ac9075-8e16-47bc-bd3d-a0a83db89d3b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:20.249371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-04ac9075-8e16-47bc-bd3d-a0a83db89d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:20.266874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa0633-a281-43dd-acca-e72ab3c5be14 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.304589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33750b3e-2b49-4d5f-9917-5970e1d1a83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.305254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33750b3e-2b49-4d5f-9917-5970e1d1a83c None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.305541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33750b3e-2b49-4d5f-9917-5970e1d1a83c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.305814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33750b3e-2b49-4d5f-9917-5970e1d1a83c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.306641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33750b3e-2b49-4d5f-9917-5970e1d1a83c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:20.308024 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33750b3e-2b49-4d5f-9917-5970e1d1a83c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:20.323405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 490/960] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:20.324285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac9075-8e16-47bc-bd3d-a0a83db89d3b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:20.387001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:20.402456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 471/961] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 12:03:20 2022] POST /identity/v3/endpoints => generated 349 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:03:20.402456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33750b3e-2b49-4d5f-9917-5970e1d1a83c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:20.453776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac9075-8e16-47bc-bd3d-a0a83db89d3b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.458163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:20.507241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ebb87-0559-455f-bd22-a2eb67fd092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.508662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ebb87-0559-455f-bd22-a2eb67fd092c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.509223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ebb87-0559-455f-bd22-a2eb67fd092c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.509961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ebb87-0559-455f-bd22-a2eb67fd092c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.525169 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ebb87-0559-455f-bd22-a2eb67fd092c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:20.527221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ebb87-0559-455f-bd22-a2eb67fd092c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:20.540092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33750b3e-2b49-4d5f-9917-5970e1d1a83c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-103ebb87-0559-455f-bd22-a2eb67fd092c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:20.550585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:20.552198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:20.553173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:20.553173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:20.553173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:20.553173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:20.557784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 491/962] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:20.558910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ebb87-0559-455f-bd22-a2eb67fd092c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:20.587271 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cd4f9-810a-4f2a-b71f-c2e2f7b49495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.588923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cd4f9-810a-4f2a-b71f-c2e2f7b49495 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.589729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cd4f9-810a-4f2a-b71f-c2e2f7b49495 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.590014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cd4f9-810a-4f2a-b71f-c2e2f7b49495 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.591024 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8cd4f9-810a-4f2a-b71f-c2e2f7b49495 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:20.592741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8cd4f9-810a-4f2a-b71f-c2e2f7b49495 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6c8cd4f9-810a-4f2a-b71f-c2e2f7b49495 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:20.610457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:20.612593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:20.613737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 472/963] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:20.617766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cd4f9-810a-4f2a-b71f-c2e2f7b49495 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:20.693703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ebb87-0559-455f-bd22-a2eb67fd092c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.704286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:20.734784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27af559-88a8-4a6d-bb58-a0aae4d0b5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.735613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27af559-88a8-4a6d-bb58-a0aae4d0b5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.735973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27af559-88a8-4a6d-bb58-a0aae4d0b5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.736326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27af559-88a8-4a6d-bb58-a0aae4d0b5b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.737162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27af559-88a8-4a6d-bb58-a0aae4d0b5b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:20.737547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27af559-88a8-4a6d-bb58-a0aae4d0b5b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:20.739177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27af559-88a8-4a6d-bb58-a0aae4d0b5b5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:20.752650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 492/964] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:20.758229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27af559-88a8-4a6d-bb58-a0aae4d0b5b5 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:20.758539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cd4f9-810a-4f2a-b71f-c2e2f7b49495 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.771363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:20.807982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c52ec-75e0-4ea8-8a73-355ac2485ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.808824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c52ec-75e0-4ea8-8a73-355ac2485ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.809247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c52ec-75e0-4ea8-8a73-355ac2485ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.809631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c52ec-75e0-4ea8-8a73-355ac2485ac6 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.811305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c52ec-75e0-4ea8-8a73-355ac2485ac6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:20.811855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c52ec-75e0-4ea8-8a73-355ac2485ac6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:20.813345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c52ec-75e0-4ea8-8a73-355ac2485ac6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:20.826644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 473/965] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:03:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:20.828821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c52ec-75e0-4ea8-8a73-355ac2485ac6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:20.898149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27af559-88a8-4a6d-bb58-a0aae4d0b5b5 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.938595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c477807-df3c-452a-9256-11c8cc338e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.940087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c477807-df3c-452a-9256-11c8cc338e87 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.940972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c477807-df3c-452a-9256-11c8cc338e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.941764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c477807-df3c-452a-9256-11c8cc338e87 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.946641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c52ec-75e0-4ea8-8a73-355ac2485ac6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:20.953452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c477807-df3c-452a-9256-11c8cc338e87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:20.955935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c477807-df3c-452a-9256-11c8cc338e87 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:20.976224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-7c477807-df3c-452a-9256-11c8cc338e87 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:20.979679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:20.980625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 12:03:20.981620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:20.988605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 493/966] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:03:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:20.989634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c477807-df3c-452a-9256-11c8cc338e87 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:20.994676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a2958-7e8c-4c1d-a539-56202eb0a317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:20.995985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a2958-7e8c-4c1d-a539-56202eb0a317 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:20.995985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a2958-7e8c-4c1d-a539-56202eb0a317 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:20.996163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a2958-7e8c-4c1d-a539-56202eb0a317 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:20.996852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a2958-7e8c-4c1d-a539-56202eb0a317 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:20.997168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a2958-7e8c-4c1d-a539-56202eb0a317 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:20.999794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a2958-7e8c-4c1d-a539-56202eb0a317 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:21.010751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 474/967] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:03:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:21.064598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687396c2-47ed-43ce-a254-f30a93860636 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:21.064960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687396c2-47ed-43ce-a254-f30a93860636 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:21.065168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687396c2-47ed-43ce-a254-f30a93860636 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:21.066958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 475/968] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:21.072691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9a0ac-f1ea-4022-9f85-beba4268f016 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:21.073055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9a0ac-f1ea-4022-9f85-beba4268f016 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:21.073246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d9a0ac-f1ea-4022-9f85-beba4268f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:21.082487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:21.092062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-98d9a0ac-f1ea-4022-9f85-beba4268f016 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:21.093275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-98d9a0ac-f1ea-4022-9f85-beba4268f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:21.144694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c477807-df3c-452a-9256-11c8cc338e87 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:21.184332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089282df-10c1-46a0-ab27-15cba9b3dce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:21.184961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089282df-10c1-46a0-ab27-15cba9b3dce8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:21.185250 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089282df-10c1-46a0-ab27-15cba9b3dce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:21.185532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089282df-10c1-46a0-ab27-15cba9b3dce8 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:21.186662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089282df-10c1-46a0-ab27-15cba9b3dce8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:21.186787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089282df-10c1-46a0-ab27-15cba9b3dce8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:21.188223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089282df-10c1-46a0-ab27-15cba9b3dce8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:21.197988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 476/969] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:21.210210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 494/970] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 19 12:03:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:21.217164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9a0ac-f1ea-4022-9f85-beba4268f016 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:21.227489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089282df-10c1-46a0-ab27-15cba9b3dce8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:21.350337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089282df-10c1-46a0-ab27-15cba9b3dce8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:21.355469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d9a0ac-f1ea-4022-9f85-beba4268f016 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:21.380405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:21.388352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:21.393555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:21.395407 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:21.395407 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:21.395407 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:21.396093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:21.397690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:21.401440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3792da-75a7-4e25-b3e3-81cb10c98946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:21.402213 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3792da-75a7-4e25-b3e3-81cb10c98946 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:21.402589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3792da-75a7-4e25-b3e3-81cb10c98946 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:21.402933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3792da-75a7-4e25-b3e3-81cb10c98946 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:21.453374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.sql.core [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 12:03:21.453374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 12:03:21.453374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [parameters: {'user_id': 'b8ba40b06ab94eca9eb1ef3042601612', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 12:03:21.453374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80603) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 19 12:03:21.460833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 12:03:21.465034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application yield resource May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application next(self.gen) May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.rollback() May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.force_reraise() May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise self.value May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application session.commit() May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 12:03:21.466243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._prepare_impl() May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.session.flush() May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._flush(objects) May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application compat.raise_( May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application flush_context.execute() May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rec.execute(self) May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 12:03:21.467605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 12:03:21.468705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result = self._query(query) May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application conn.query(q) May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application result.read() May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 12:03:21.469770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b8ba40b06ab94eca9eb1ef3042601612', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:21.470806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 12:03:21.471773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 12:03:21.472848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 12:03:21.472848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 12:03:21.472848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 12:03:21.472848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 12:03:21.472848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 12:03:21.472848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:21.472848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 495/971] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 19 12:03:21 2022] POST /identity/v3/users => generated 160 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:21.474436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:21.501409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3792da-75a7-4e25-b3e3-81cb10c98946 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:21.503493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3792da-75a7-4e25-b3e3-81cb10c98946 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:21.612173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc20e7-6283-4f3c-b0e4-c5c3384d3b04 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:21.630325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 477/972] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:21 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 12:03:21.631810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3792da-75a7-4e25-b3e3-81cb10c98946 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:21.663910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0390e-da36-486d-a8b3-c9ac9e28e8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:21.664725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0390e-da36-486d-a8b3-c9ac9e28e8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:21.665008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0390e-da36-486d-a8b3-c9ac9e28e8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:21.665286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0390e-da36-486d-a8b3-c9ac9e28e8c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:21.668433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:21.677356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0390e-da36-486d-a8b3-c9ac9e28e8c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:21.679300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0390e-da36-486d-a8b3-c9ac9e28e8c6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-0cc0390e-da36-486d-a8b3-c9ac9e28e8c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:21.702445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:21.703568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:21.704740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 496/973] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:03:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:21.706762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0390e-da36-486d-a8b3-c9ac9e28e8c6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:21.754286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3792da-75a7-4e25-b3e3-81cb10c98946 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:21.771554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:21.792417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95598ddf-e60b-42c1-8e33-44c2f19cbdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:21.795992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95598ddf-e60b-42c1-8e33-44c2f19cbdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:21.795992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95598ddf-e60b-42c1-8e33-44c2f19cbdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:21.795992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95598ddf-e60b-42c1-8e33-44c2f19cbdf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:21.807674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95598ddf-e60b-42c1-8e33-44c2f19cbdf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:21.809631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95598ddf-e60b-42c1-8e33-44c2f19cbdf9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-95598ddf-e60b-42c1-8e33-44c2f19cbdf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:21.833499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:21.834681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:21.835617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:21.835617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:21.835617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:21.835617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:21.835839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0390e-da36-486d-a8b3-c9ac9e28e8c6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:21.835914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 478/974] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:21.841662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95598ddf-e60b-42c1-8e33-44c2f19cbdf9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:21.880224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5d7d7-144d-4e57-9439-50689d2194f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:21.881118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5d7d7-144d-4e57-9439-50689d2194f8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:21.881408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5d7d7-144d-4e57-9439-50689d2194f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:21.881690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5d7d7-144d-4e57-9439-50689d2194f8 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:21.883206 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5d7d7-144d-4e57-9439-50689d2194f8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:21.883524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5d7d7-144d-4e57-9439-50689d2194f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:21.885863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5d7d7-144d-4e57-9439-50689d2194f8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:21.936375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:21.942177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 497/975] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 19 12:03:21 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:21.968449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95598ddf-e60b-42c1-8e33-44c2f19cbdf9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:22.009742 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3fb52-e1d1-45e5-9aec-12842acb27fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:22.010463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3fb52-e1d1-45e5-9aec-12842acb27fa None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:22.010862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3fb52-e1d1-45e5-9aec-12842acb27fa None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:22.011143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3fb52-e1d1-45e5-9aec-12842acb27fa None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:22.011884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3fb52-e1d1-45e5-9aec-12842acb27fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:22.012192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3fb52-e1d1-45e5-9aec-12842acb27fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:22.013580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3fb52-e1d1-45e5-9aec-12842acb27fa None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:22.027316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 479/976] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:22.032679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5d7d7-144d-4e57-9439-50689d2194f8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:22.156619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5d7d7-144d-4e57-9439-50689d2194f8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:22.159879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:22.194218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3170-c9d2-486e-88be-32623a022219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:22.195079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3170-c9d2-486e-88be-32623a022219 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:22.195363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3170-c9d2-486e-88be-32623a022219 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:22.195637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db3170-c9d2-486e-88be-32623a022219 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:22.204175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db3170-c9d2-486e-88be-32623a022219 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:22.205627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db3170-c9d2-486e-88be-32623a022219 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-79db3170-c9d2-486e-88be-32623a022219 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:22.220926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:22.221915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:22.223007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:22.223007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:22.223007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:22.223007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:22.223007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:22.223007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:22.223007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:22.223007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:22.223007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 498/977] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:22.230936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3fb52-e1d1-45e5-9aec-12842acb27fa None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:22.335953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3fb52-e1d1-45e5-9aec-12842acb27fa None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:22.371405 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a920449-f5ee-454f-98a4-37871336d54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:22.372068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a920449-f5ee-454f-98a4-37871336d54f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:22.372349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a920449-f5ee-454f-98a4-37871336d54f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:22.372618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a920449-f5ee-454f-98a4-37871336d54f None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:22.373405 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a920449-f5ee-454f-98a4-37871336d54f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:22.373711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a920449-f5ee-454f-98a4-37871336d54f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:22.375166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a920449-f5ee-454f-98a4-37871336d54f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:22.376822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:22.388940 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 480/978] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:22.394692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3170-c9d2-486e-88be-32623a022219 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:22.497060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db3170-c9d2-486e-88be-32623a022219 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:22.531952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c3577-ad6f-4b07-acc2-290ca19f43a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:22.532610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c3577-ad6f-4b07-acc2-290ca19f43a7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:22.532892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c3577-ad6f-4b07-acc2-290ca19f43a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:22.533169 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c3577-ad6f-4b07-acc2-290ca19f43a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:22.534007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c3577-ad6f-4b07-acc2-290ca19f43a7 None admin] RBAC: Adding query filter params (role.id=d538a3ae543f4c1897c8c3d9c8fb8066, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=6be447727ed44a48a6b023303289e319) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:22.534307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c3577-ad6f-4b07-acc2-290ca19f43a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:22.535768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c3577-ad6f-4b07-acc2-290ca19f43a7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:22.550670 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 499/979] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:22 2022] GET /identity/v3/role_assignments?user.id=6be447727ed44a48a6b023303289e319&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=d538a3ae543f4c1897c8c3d9c8fb8066 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:22.819153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0702-b375-4bb3-a9d8-ff17f7c31e20 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:22.819661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0702-b375-4bb3-a9d8-ff17f7c31e20 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:22.819661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0702-b375-4bb3-a9d8-ff17f7c31e20 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:22.821132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 481/980] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:22.827099 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807a224-30cd-43c2-af56-77de8a8ccc39 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:22.827530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807a224-30cd-43c2-af56-77de8a8ccc39 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:22.827714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807a224-30cd-43c2-af56-77de8a8ccc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:22.844594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6807a224-30cd-43c2-af56-77de8a8ccc39 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:22.846112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6807a224-30cd-43c2-af56-77de8a8ccc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:22.870161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:22.932335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 500/981] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:22.955336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0702-b375-4bb3-a9d8-ff17f7c31e20 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:23.067568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0702-b375-4bb3-a9d8-ff17f7c31e20 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:23.106507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5e879-5d17-445b-953c-f48bdd4a2476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:23.107206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5e879-5d17-445b-953c-f48bdd4a2476 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:23.107491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5e879-5d17-445b-953c-f48bdd4a2476 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:23.107767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5e879-5d17-445b-953c-f48bdd4a2476 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:23.108423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5e879-5d17-445b-953c-f48bdd4a2476 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:23.109790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5e879-5d17-445b-953c-f48bdd4a2476 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-bab5e879-5d17-445b-953c-f48bdd4a2476 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:23.122207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:23.123285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:23.124219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 482/982] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:23.129382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807a224-30cd-43c2-af56-77de8a8ccc39 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:23.244296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807a224-30cd-43c2-af56-77de8a8ccc39 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:23.284021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cc75d-c921-4a0a-9fc4-0c59b14d6d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:23.284689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cc75d-c921-4a0a-9fc4-0c59b14d6d2b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:23.284968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cc75d-c921-4a0a-9fc4-0c59b14d6d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:23.285240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cc75d-c921-4a0a-9fc4-0c59b14d6d2b None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:23.285940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cc75d-c921-4a0a-9fc4-0c59b14d6d2b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:23.286246 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cc75d-c921-4a0a-9fc4-0c59b14d6d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:23.287632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cc75d-c921-4a0a-9fc4-0c59b14d6d2b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:23.294636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 501/983] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:23.299446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5e879-5d17-445b-953c-f48bdd4a2476 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:23.361805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:23.371522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d3eb6-6903-4491-97f1-6a1344e02964 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:23.371522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d3eb6-6903-4491-97f1-6a1344e02964 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:23.372745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d3eb6-6903-4491-97f1-6a1344e02964 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:23.374265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 502/984] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:23.379437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769dbc1-86d7-4e1c-9a13-60c3a9e14bc2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:23.379730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769dbc1-86d7-4e1c-9a13-60c3a9e14bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:23.380012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769dbc1-86d7-4e1c-9a13-60c3a9e14bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:23.397898 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f769dbc1-86d7-4e1c-9a13-60c3a9e14bc2 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:23.399114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f769dbc1-86d7-4e1c-9a13-60c3a9e14bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:23.410514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5e879-5d17-445b-953c-f48bdd4a2476 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:23.418666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:23.449511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4cfe3-3b14-4e7d-abf4-6b21ba334272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:23.450210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4cfe3-3b14-4e7d-abf4-6b21ba334272 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:23.450525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4cfe3-3b14-4e7d-abf4-6b21ba334272 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:23.450803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4cfe3-3b14-4e7d-abf4-6b21ba334272 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:23.451513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4cfe3-3b14-4e7d-abf4-6b21ba334272 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:23.451917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4cfe3-3b14-4e7d-abf4-6b21ba334272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:23.453283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4cfe3-3b14-4e7d-abf4-6b21ba334272 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:23.458481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 483/985] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:23.464529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4cfe3-3b14-4e7d-abf4-6b21ba334272 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:23.474325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 503/986] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:23.497700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769dbc1-86d7-4e1c-9a13-60c3a9e14bc2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:23.582596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4cfe3-3b14-4e7d-abf4-6b21ba334272 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:23.602591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769dbc1-86d7-4e1c-9a13-60c3a9e14bc2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:23.610390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:23.623103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2c865-ee93-4030-b487-18e7f882eeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:23.623700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2c865-ee93-4030-b487-18e7f882eeaa None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:23.623983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2c865-ee93-4030-b487-18e7f882eeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:23.624682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2c865-ee93-4030-b487-18e7f882eeaa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:23.625490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2c865-ee93-4030-b487-18e7f882eeaa None admin] RBAC: Adding query filter params (interface=admin, service_id=f89669e4112d4b6f968836e5622c7a47, region_id=RegionOne) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:23.625809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2c865-ee93-4030-b487-18e7f882eeaa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:23.627195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2c865-ee93-4030-b487-18e7f882eeaa None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:23.635260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 484/987] 10.222.0.57 () {56 vars in 1249 bytes} [Thu May 19 12:03:23 2022] GET /identity/v3/endpoints?service_id=f89669e4112d4b6f968836e5622c7a47&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:23.638702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73c2cc-d9f9-4405-b6ad-c673bcf6ad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:23.639547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73c2cc-d9f9-4405-b6ad-c673bcf6ad9d None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:23.639653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73c2cc-d9f9-4405-b6ad-c673bcf6ad9d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:23.639910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73c2cc-d9f9-4405-b6ad-c673bcf6ad9d None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:23.640379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2c865-ee93-4030-b487-18e7f882eeaa None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:23.640630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73c2cc-d9f9-4405-b6ad-c673bcf6ad9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:23.642100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73c2cc-d9f9-4405-b6ad-c673bcf6ad9d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:23.722213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 504/988] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:03:23 2022] POST /identity/v3/services => generated 242 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:03:23.745299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2c865-ee93-4030-b487-18e7f882eeaa None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:23.794213 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66471564-f9e0-4f61-91be-dcf71c93a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:23.794935 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66471564-f9e0-4f61-91be-dcf71c93a474 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:23.795220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66471564-f9e0-4f61-91be-dcf71c93a474 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:23.795517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66471564-f9e0-4f61-91be-dcf71c93a474 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:23.796270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66471564-f9e0-4f61-91be-dcf71c93a474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:23.797660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66471564-f9e0-4f61-91be-dcf71c93a474 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:23.810442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 485/989] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 12:03:23 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:03:24.937361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbd335-dd52-4421-ad55-9c5bf14c87f0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:24.937931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbd335-dd52-4421-ad55-9c5bf14c87f0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:24.938495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbd335-dd52-4421-ad55-9c5bf14c87f0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:24.941001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 505/990] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:24.949251 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715727c-87fe-4b0b-93b2-941ac8a991a9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:24.949767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715727c-87fe-4b0b-93b2-941ac8a991a9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:24.950194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715727c-87fe-4b0b-93b2-941ac8a991a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:24.970659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1715727c-87fe-4b0b-93b2-941ac8a991a9 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:24.972009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1715727c-87fe-4b0b-93b2-941ac8a991a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:25.030771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:25.056029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 486/991] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:24 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:25.077863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbd335-dd52-4421-ad55-9c5bf14c87f0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:25.093525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:25.182757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbd335-dd52-4421-ad55-9c5bf14c87f0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:25.216522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156651e2-2065-4c62-975d-3c2a4fbae908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:25.217224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156651e2-2065-4c62-975d-3c2a4fbae908 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:25.217508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156651e2-2065-4c62-975d-3c2a4fbae908 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:25.217782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156651e2-2065-4c62-975d-3c2a4fbae908 None admin] PATH_INFO: `/v3/services/image` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:25.218449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156651e2-2065-4c62-975d-3c2a4fbae908 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:25.219811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156651e2-2065-4c62-975d-3c2a4fbae908 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-156651e2-2065-4c62-975d-3c2a4fbae908 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:25.231733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:25.232829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 12:03:25.233684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:25.237773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 506/992] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 12:03:25 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:25.238789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715727c-87fe-4b0b-93b2-941ac8a991a9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:25.345804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715727c-87fe-4b0b-93b2-941ac8a991a9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:25.380393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd7a89-db47-4d8b-85de-bdaf9baaa6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:25.381053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7a89-db47-4d8b-85de-bdaf9baaa6da None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:25.381336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7a89-db47-4d8b-85de-bdaf9baaa6da None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:25.381610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd7a89-db47-4d8b-85de-bdaf9baaa6da None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:25.382307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd7a89-db47-4d8b-85de-bdaf9baaa6da None admin] RBAC: Adding query filter params (name=image) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:25.382643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd7a89-db47-4d8b-85de-bdaf9baaa6da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:25.384019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd7a89-db47-4d8b-85de-bdaf9baaa6da None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:25.389340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 487/993] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:25.394854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156651e2-2065-4c62-975d-3c2a4fbae908 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:25.473989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:25.496747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156651e2-2065-4c62-975d-3c2a4fbae908 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:25.531681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61802cf4-e7bb-403b-ba07-821bdd8c3343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:25.532719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61802cf4-e7bb-403b-ba07-821bdd8c3343 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:25.533209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61802cf4-e7bb-403b-ba07-821bdd8c3343 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:25.533707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61802cf4-e7bb-403b-ba07-821bdd8c3343 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:25.534655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61802cf4-e7bb-403b-ba07-821bdd8c3343 None admin] RBAC: Adding query filter params (type=image) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:25.535184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61802cf4-e7bb-403b-ba07-821bdd8c3343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:25.536814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61802cf4-e7bb-403b-ba07-821bdd8c3343 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:25.542900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 507/994] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:26.060656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde29ac-86ff-4413-afbc-b13eb14e1136 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.061161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde29ac-86ff-4413-afbc-b13eb14e1136 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.061272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde29ac-86ff-4413-afbc-b13eb14e1136 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.062778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 488/995] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:26.068007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997fc55-5945-44bc-86ed-f6b1f6a59e19 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.068415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997fc55-5945-44bc-86ed-f6b1f6a59e19 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.068591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997fc55-5945-44bc-86ed-f6b1f6a59e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.086319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f997fc55-5945-44bc-86ed-f6b1f6a59e19 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:26.087559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f997fc55-5945-44bc-86ed-f6b1f6a59e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:26.130195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb94874-e24b-407c-9ec4-b75e3cd7bb85 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.130507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb94874-e24b-407c-9ec4-b75e3cd7bb85 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.130793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb94874-e24b-407c-9ec4-b75e3cd7bb85 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.132299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 489/996] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:26.137804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee13b2-92ba-4ead-a70a-fc378c74650c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.138099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee13b2-92ba-4ead-a70a-fc378c74650c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.138386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee13b2-92ba-4ead-a70a-fc378c74650c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.156621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-47ee13b2-92ba-4ead-a70a-fc378c74650c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:26.158141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-47ee13b2-92ba-4ead-a70a-fc378c74650c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:26.158855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 508/997] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:26.182456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997fc55-5945-44bc-86ed-f6b1f6a59e19 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:26.202277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:26.237294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 490/998] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:26.260425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:26.260705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee13b2-92ba-4ead-a70a-fc378c74650c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:26.294481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997fc55-5945-44bc-86ed-f6b1f6a59e19 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:26.331559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0545b2-5019-4ab8-8ded-52ea21f8ac46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:26.332368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0545b2-5019-4ab8-8ded-52ea21f8ac46 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.332659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0545b2-5019-4ab8-8ded-52ea21f8ac46 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.332945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0545b2-5019-4ab8-8ded-52ea21f8ac46 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.333611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0545b2-5019-4ab8-8ded-52ea21f8ac46 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:26.335068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0545b2-5019-4ab8-8ded-52ea21f8ac46 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-9a0545b2-5019-4ab8-8ded-52ea21f8ac46 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:26.347093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:26.348143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:26.349004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 509/999] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:26.355216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0545b2-5019-4ab8-8ded-52ea21f8ac46 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:26.370627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee13b2-92ba-4ead-a70a-fc378c74650c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:26.407459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972399bc-558d-4e3c-91d8-47b8bbb28378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:26.408262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972399bc-558d-4e3c-91d8-47b8bbb28378 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.408550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972399bc-558d-4e3c-91d8-47b8bbb28378 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.408821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972399bc-558d-4e3c-91d8-47b8bbb28378 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.409485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972399bc-558d-4e3c-91d8-47b8bbb28378 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:26.410897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972399bc-558d-4e3c-91d8-47b8bbb28378 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-972399bc-558d-4e3c-91d8-47b8bbb28378 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:26.424018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:26.424994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:26.425830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 491/1000] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 19 12:03:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:26.431760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972399bc-558d-4e3c-91d8-47b8bbb28378 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:26.463030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0545b2-5019-4ab8-8ded-52ea21f8ac46 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:26.469651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:26.500833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4fa3e-ee8a-4a78-a909-08213c0c80eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:26.501866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4fa3e-ee8a-4a78-a909-08213c0c80eb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.502430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4fa3e-ee8a-4a78-a909-08213c0c80eb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.502911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4fa3e-ee8a-4a78-a909-08213c0c80eb None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.503808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4fa3e-ee8a-4a78-a909-08213c0c80eb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:26.504331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4fa3e-ee8a-4a78-a909-08213c0c80eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:26.505897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4fa3e-ee8a-4a78-a909-08213c0c80eb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:26.507368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:26.515519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 510/1001] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:03:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:26.517675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4fa3e-ee8a-4a78-a909-08213c0c80eb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:26.542028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972399bc-558d-4e3c-91d8-47b8bbb28378 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:26.579875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e1710-54ff-4330-b0fd-faaa63c8c254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:26.580456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e1710-54ff-4330-b0fd-faaa63c8c254 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.580736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e1710-54ff-4330-b0fd-faaa63c8c254 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.581010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e1710-54ff-4330-b0fd-faaa63c8c254 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.581713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e1710-54ff-4330-b0fd-faaa63c8c254 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:26.582019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e1710-54ff-4330-b0fd-faaa63c8c254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:26.583431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e1710-54ff-4330-b0fd-faaa63c8c254 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:26.589929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 492/1002] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:26.595320 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e1710-54ff-4330-b0fd-faaa63c8c254 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:26.630311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4fa3e-ee8a-4a78-a909-08213c0c80eb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:26.669130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a21a3e-d33b-4dc9-8ebf-d273f43a2faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:26.669812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a21a3e-d33b-4dc9-8ebf-d273f43a2faf None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.670112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a21a3e-d33b-4dc9-8ebf-d273f43a2faf None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.670371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a21a3e-d33b-4dc9-8ebf-d273f43a2faf None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.671133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a21a3e-d33b-4dc9-8ebf-d273f43a2faf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:26.671442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a21a3e-d33b-4dc9-8ebf-d273f43a2faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:26.672863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a21a3e-d33b-4dc9-8ebf-d273f43a2faf None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:26.679069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 511/1003] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:03:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:26.685507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a21a3e-d33b-4dc9-8ebf-d273f43a2faf None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:26.700780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:26.705535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e1710-54ff-4330-b0fd-faaa63c8c254 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:26.745432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8084cd-a03c-4970-905e-a8ee6bb7f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:26.746108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8084cd-a03c-4970-905e-a8ee6bb7f672 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.746545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8084cd-a03c-4970-905e-a8ee6bb7f672 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.747220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8084cd-a03c-4970-905e-a8ee6bb7f672 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.748607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8084cd-a03c-4970-905e-a8ee6bb7f672 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:26.749453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8084cd-a03c-4970-905e-a8ee6bb7f672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:26.751941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8084cd-a03c-4970-905e-a8ee6bb7f672 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:26.756529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:26.759878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 493/1004] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 19 12:03:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:26.765505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8084cd-a03c-4970-905e-a8ee6bb7f672 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:26.801370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a21a3e-d33b-4dc9-8ebf-d273f43a2faf None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:26.837796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565b3e9-435a-4f61-82e1-445a859b124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:26.838473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565b3e9-435a-4f61-82e1-445a859b124e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.838767 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565b3e9-435a-4f61-82e1-445a859b124e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.839072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565b3e9-435a-4f61-82e1-445a859b124e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.839845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565b3e9-435a-4f61-82e1-445a859b124e None admin] RBAC: Adding query filter params (interface=public, service_id=d0951f627d5741fe92c25a540859a0be, region_id=RegionOne) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:26.840165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565b3e9-435a-4f61-82e1-445a859b124e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:26.841538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565b3e9-435a-4f61-82e1-445a859b124e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:26.846803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 512/1005] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 12:03:26 2022] GET /identity/v3/endpoints?service_id=d0951f627d5741fe92c25a540859a0be&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:26.855441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565b3e9-435a-4f61-82e1-445a859b124e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:26.885363 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8084cd-a03c-4970-905e-a8ee6bb7f672 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:26.921527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e86a81-d20f-4964-be91-a09a16559ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:26.922191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e86a81-d20f-4964-be91-a09a16559ed0 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.922504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e86a81-d20f-4964-be91-a09a16559ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.922786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e86a81-d20f-4964-be91-a09a16559ed0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:26.923574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e86a81-d20f-4964-be91-a09a16559ed0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:26.924936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e86a81-d20f-4964-be91-a09a16559ed0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:26.963639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565b3e9-435a-4f61-82e1-445a859b124e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:26.989284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:26.998559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9c473-4d31-4a03-9689-641fd7f2215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:26.999287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9c473-4d31-4a03-9689-641fd7f2215f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:26.999579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9c473-4d31-4a03-9689-641fd7f2215f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:26.999861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9c473-4d31-4a03-9689-641fd7f2215f None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:27.000541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9c473-4d31-4a03-9689-641fd7f2215f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:27.001910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9c473-4d31-4a03-9689-641fd7f2215f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:27.008106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 513/1006] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 12:03:26 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:03:27.064648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 494/1007] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 12:03:26 2022] POST /identity/v3/endpoints => generated 325 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:03:27.799885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37725ee-747e-4840-b683-c12d669a3ff4 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:27.800429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37725ee-747e-4840-b683-c12d669a3ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:27.800509 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37725ee-747e-4840-b683-c12d669a3ff4 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:27.801996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 514/1008] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:27.807774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4084d62-046d-4cfd-96bd-8b4b3feaf086 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:27.808062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4084d62-046d-4cfd-96bd-8b4b3feaf086 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:27.808338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4084d62-046d-4cfd-96bd-8b4b3feaf086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:27.825286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d4084d62-046d-4cfd-96bd-8b4b3feaf086 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:27.826490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d4084d62-046d-4cfd-96bd-8b4b3feaf086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:27.843631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:27.900685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 495/1009] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:27 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:27.923858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37725ee-747e-4840-b683-c12d669a3ff4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:28.033878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37725ee-747e-4840-b683-c12d669a3ff4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:28.067777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e59169-30a2-49b1-bbd1-5edc95deac8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:28.068359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59169-30a2-49b1-bbd1-5edc95deac8d None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:28.068636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59169-30a2-49b1-bbd1-5edc95deac8d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:28.068908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e59169-30a2-49b1-bbd1-5edc95deac8d None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:28.069658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e59169-30a2-49b1-bbd1-5edc95deac8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:28.071140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e59169-30a2-49b1-bbd1-5edc95deac8d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:28.198143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 515/1010] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:03:27 2022] POST /identity/v3/services => generated 242 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:03:29.346000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef2412-9552-4734-aaeb-acfb4a208fd0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:29.346520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef2412-9552-4734-aaeb-acfb4a208fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:29.346520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef2412-9552-4734-aaeb-acfb4a208fd0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:29.348011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 496/1011] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:29.353423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d86d6d-5e3a-49ea-a646-0202b0ff784c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:29.353860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d86d6d-5e3a-49ea-a646-0202b0ff784c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:29.354027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d86d6d-5e3a-49ea-a646-0202b0ff784c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:29.371393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-59d86d6d-5e3a-49ea-a646-0202b0ff784c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:29.372631 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-59d86d6d-5e3a-49ea-a646-0202b0ff784c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:29.388570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5e559-a9e3-458f-8ffe-cfe2ef50ecc4 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:29.388855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5e559-a9e3-458f-8ffe-cfe2ef50ecc4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:29.389305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5e559-a9e3-458f-8ffe-cfe2ef50ecc4 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:29.390706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 497/1012] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:29.396062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b4a0-d24d-44fa-89cf-4ff95ac6e743 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:29.396346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b4a0-d24d-44fa-89cf-4ff95ac6e743 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:29.396826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b4a0-d24d-44fa-89cf-4ff95ac6e743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:29.414022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e2c2b4a0-d24d-44fa-89cf-4ff95ac6e743 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:29.416645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e2c2b4a0-d24d-44fa-89cf-4ff95ac6e743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:29.427193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:29.456685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 516/1013] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:29 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:29.479732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d86d6d-5e3a-49ea-a646-0202b0ff784c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:29.488965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 498/1014] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:29 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:29.512525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b4a0-d24d-44fa-89cf-4ff95ac6e743 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:29.590942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d86d6d-5e3a-49ea-a646-0202b0ff784c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:29.594906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:29.619804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b4a0-d24d-44fa-89cf-4ff95ac6e743 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:29.632310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c4caf-57f1-4175-9c92-3c1e2af3f19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:29.633118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c4caf-57f1-4175-9c92-3c1e2af3f19e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:29.634030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c4caf-57f1-4175-9c92-3c1e2af3f19e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:29.634440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c4caf-57f1-4175-9c92-3c1e2af3f19e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:29.635192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c4caf-57f1-4175-9c92-3c1e2af3f19e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:29.636575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c4caf-57f1-4175-9c92-3c1e2af3f19e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a76c4caf-57f1-4175-9c92-3c1e2af3f19e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:29.652492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:29.653727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:29.654806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 517/1015] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:29.658742 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d993a67-6d35-4017-9a53-637d90a962d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:29.659646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d993a67-6d35-4017-9a53-637d90a962d5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:29.659938 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d993a67-6d35-4017-9a53-637d90a962d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:29.660122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c4caf-57f1-4175-9c92-3c1e2af3f19e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:29.660233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d993a67-6d35-4017-9a53-637d90a962d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:29.661499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d993a67-6d35-4017-9a53-637d90a962d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:29.663307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d993a67-6d35-4017-9a53-637d90a962d5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:29.669316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 499/1016] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:29.673130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d993a67-6d35-4017-9a53-637d90a962d5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:29.711294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:29.782440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c4caf-57f1-4175-9c92-3c1e2af3f19e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:29.783507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d993a67-6d35-4017-9a53-637d90a962d5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:29.815968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57f553-ca12-485e-bc3f-adb51e11bc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:29.816925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57f553-ca12-485e-bc3f-adb51e11bc6c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:29.817209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57f553-ca12-485e-bc3f-adb51e11bc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:29.817482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57f553-ca12-485e-bc3f-adb51e11bc6c None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:29.818187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57f553-ca12-485e-bc3f-adb51e11bc6c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:29.818526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57f553-ca12-485e-bc3f-adb51e11bc6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:29.819882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57f553-ca12-485e-bc3f-adb51e11bc6c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:29.819984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b31a49-253d-4fff-b956-0afb20053e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:29.820640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b31a49-253d-4fff-b956-0afb20053e69 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:29.821094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b31a49-253d-4fff-b956-0afb20053e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:29.822041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b31a49-253d-4fff-b956-0afb20053e69 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:29.822852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b31a49-253d-4fff-b956-0afb20053e69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:29.824557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b31a49-253d-4fff-b956-0afb20053e69 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:29.825294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 518/1017] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:03:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:29.830727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57f553-ca12-485e-bc3f-adb51e11bc6c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:29.922486 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:29.933042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57f553-ca12-485e-bc3f-adb51e11bc6c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:29.939775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 500/1018] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:03:29 2022] POST /identity/v3/projects => generated 311 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:03:29.944046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:29.980196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ad3a2-44ad-4168-8d8b-8d38f59f33a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:29.980883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad3a2-44ad-4168-8d8b-8d38f59f33a1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:29.981167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad3a2-44ad-4168-8d8b-8d38f59f33a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:29.981457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad3a2-44ad-4168-8d8b-8d38f59f33a1 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:29.982169 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ad3a2-44ad-4168-8d8b-8d38f59f33a1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:29.982516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ad3a2-44ad-4168-8d8b-8d38f59f33a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:29.983920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ad3a2-44ad-4168-8d8b-8d38f59f33a1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:29.994805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 519/1019] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 19 12:03:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:29.995988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b31a49-253d-4fff-b956-0afb20053e69 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:30.122319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b31a49-253d-4fff-b956-0afb20053e69 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:30.163254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b6f07-7290-479e-b557-43b024a7df2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:30.163921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b6f07-7290-479e-b557-43b024a7df2e None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:30.164207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b6f07-7290-479e-b557-43b024a7df2e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:30.164491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b6f07-7290-479e-b557-43b024a7df2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:30.166054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b6f07-7290-479e-b557-43b024a7df2e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:30.167726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b6f07-7290-479e-b557-43b024a7df2e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:30.278111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 501/1020] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 12:03:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:03:30.524997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fe820-5b8f-46e5-a85c-2a0bc8561158 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:30.525514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fe820-5b8f-46e5-a85c-2a0bc8561158 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:30.525680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fe820-5b8f-46e5-a85c-2a0bc8561158 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:30.526976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 520/1021] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:30.532806 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6d547-e965-40b1-8402-8673972df2b5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:30.533586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6d547-e965-40b1-8402-8673972df2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:30.533869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6d547-e965-40b1-8402-8673972df2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:30.552812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7ca6d547-e965-40b1-8402-8673972df2b5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:30.554022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7ca6d547-e965-40b1-8402-8673972df2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:30.608896 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:30.638433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 502/1022] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:30.657117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fe820-5b8f-46e5-a85c-2a0bc8561158 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:30.764542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fe820-5b8f-46e5-a85c-2a0bc8561158 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:30.807544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489fceb-990b-43e3-ac90-4d135263cdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:30.808571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489fceb-990b-43e3-ac90-4d135263cdfa None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:30.808571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489fceb-990b-43e3-ac90-4d135263cdfa None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:30.808891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489fceb-990b-43e3-ac90-4d135263cdfa None admin] PATH_INFO: `/v3/services/image` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:30.809465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489fceb-990b-43e3-ac90-4d135263cdfa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:30.811041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489fceb-990b-43e3-ac90-4d135263cdfa None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a489fceb-990b-43e3-ac90-4d135263cdfa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:30.824052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:30.825056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:30.830996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 521/1023] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 12:03:30 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:30.833704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6d547-e965-40b1-8402-8673972df2b5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:30.942296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6d547-e965-40b1-8402-8673972df2b5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:30.976347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77f4c3-de98-4040-a508-5bf9e8ad4b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:30.977015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77f4c3-de98-4040-a508-5bf9e8ad4b1f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:30.977301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77f4c3-de98-4040-a508-5bf9e8ad4b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:30.978115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77f4c3-de98-4040-a508-5bf9e8ad4b1f None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:30.979028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77f4c3-de98-4040-a508-5bf9e8ad4b1f None admin] RBAC: Adding query filter params (name=image) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:30.979352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77f4c3-de98-4040-a508-5bf9e8ad4b1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:30.980712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77f4c3-de98-4040-a508-5bf9e8ad4b1f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:30.985796 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 503/1024] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:30.991449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489fceb-990b-43e3-ac90-4d135263cdfa None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:31.029325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:31.101446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489fceb-990b-43e3-ac90-4d135263cdfa None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:31.136702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2a1de-d5cc-4d91-b8b8-7ebee182f8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:31.137468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2a1de-d5cc-4d91-b8b8-7ebee182f8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:31.137755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2a1de-d5cc-4d91-b8b8-7ebee182f8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:31.138033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2a1de-d5cc-4d91-b8b8-7ebee182f8d1 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:31.138804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2a1de-d5cc-4d91-b8b8-7ebee182f8d1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:31.139121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2a1de-d5cc-4d91-b8b8-7ebee182f8d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:31.140549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2a1de-d5cc-4d91-b8b8-7ebee182f8d1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:31.151649 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 522/1025] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:31.153125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77f4c3-de98-4040-a508-5bf9e8ad4b1f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:31.201170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:31.263290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77f4c3-de98-4040-a508-5bf9e8ad4b1f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:31.298751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01b533-f7f8-4701-8380-5efe43118d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:31.299413 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01b533-f7f8-4701-8380-5efe43118d63 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:31.299700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01b533-f7f8-4701-8380-5efe43118d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:31.299980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01b533-f7f8-4701-8380-5efe43118d63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:31.300772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01b533-f7f8-4701-8380-5efe43118d63 None admin] RBAC: Adding query filter params (interface=public, service_id=d5711238722d4e2fb0347ed13aaeea68, region_id=RegionOne) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:31.301092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01b533-f7f8-4701-8380-5efe43118d63 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:31.302483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01b533-f7f8-4701-8380-5efe43118d63 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:31.313899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 504/1026] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 12:03:31 2022] GET /identity/v3/endpoints?service_id=d5711238722d4e2fb0347ed13aaeea68&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:31.314116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2a1de-d5cc-4d91-b8b8-7ebee182f8d1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:31.412110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:31.419655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2a1de-d5cc-4d91-b8b8-7ebee182f8d1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:31.456206 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785ce2c-ba96-4dd3-95b5-0fb903c0e47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:31.456935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785ce2c-ba96-4dd3-95b5-0fb903c0e47a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:31.457323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785ce2c-ba96-4dd3-95b5-0fb903c0e47a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:31.457609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785ce2c-ba96-4dd3-95b5-0fb903c0e47a None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:31.458386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785ce2c-ba96-4dd3-95b5-0fb903c0e47a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:31.459852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785ce2c-ba96-4dd3-95b5-0fb903c0e47a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:31.470790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 523/1027] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 12:03:31 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:03:32.258317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c099d-96f1-4fdd-9e6a-88146d86c517 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:32.259002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c099d-96f1-4fdd-9e6a-88146d86c517 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:32.259002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c099d-96f1-4fdd-9e6a-88146d86c517 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:32.260524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 505/1028] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:32.265956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b5ad9-6464-425d-b3c7-7d7b311393dd None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:32.266624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b5ad9-6464-425d-b3c7-7d7b311393dd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:32.266624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b5ad9-6464-425d-b3c7-7d7b311393dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:32.283607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a92b5ad9-6464-425d-b3c7-7d7b311393dd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:32.284711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a92b5ad9-6464-425d-b3c7-7d7b311393dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:32.355515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 524/1029] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:32.380311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c099d-96f1-4fdd-9e6a-88146d86c517 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:32.491889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c099d-96f1-4fdd-9e6a-88146d86c517 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:32.503267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:32.529831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcc96b-9757-4e29-8f1d-9534a048efce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:32.530797 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcc96b-9757-4e29-8f1d-9534a048efce None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:32.531083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcc96b-9757-4e29-8f1d-9534a048efce None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:32.531360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcc96b-9757-4e29-8f1d-9534a048efce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:32.532607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcc96b-9757-4e29-8f1d-9534a048efce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:32.534099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcc96b-9757-4e29-8f1d-9534a048efce None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:32.535879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 506/1030] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:32.543169 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b5ad9-6464-425d-b3c7-7d7b311393dd None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:32.660073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:32.666864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7407c30-c583-41d9-8e8e-d68ec0dd0f94 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:32.667159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7407c30-c583-41d9-8e8e-d68ec0dd0f94 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:32.667443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7407c30-c583-41d9-8e8e-d68ec0dd0f94 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:32.668989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 507/1031] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:32.671212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b5ad9-6464-425d-b3c7-7d7b311393dd None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:32.675917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ff118-5e21-44ff-95c3-2ce449b004a6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:32.675917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ff118-5e21-44ff-95c3-2ce449b004a6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:32.675917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ff118-5e21-44ff-95c3-2ce449b004a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:32.693338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a32ff118-5e21-44ff-95c3-2ce449b004a6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:32.694558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a32ff118-5e21-44ff-95c3-2ce449b004a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:32.713443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2b648-01d1-407c-a60d-2c289ef5b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:32.714848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2b648-01d1-407c-a60d-2c289ef5b31b None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:32.715340 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2b648-01d1-407c-a60d-2c289ef5b31b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:32.715813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2b648-01d1-407c-a60d-2c289ef5b31b None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:32.716740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2b648-01d1-407c-a60d-2c289ef5b31b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:32.719250 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2b648-01d1-407c-a60d-2c289ef5b31b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:32.766555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 508/1032] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:32.790738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ff118-5e21-44ff-95c3-2ce449b004a6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:32.866570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 525/1033] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:03:32 2022] POST /identity/v3/users => generated 285 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:03:32.910675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ff118-5e21-44ff-95c3-2ce449b004a6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:32.919607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:32.951703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ff8ea-25cd-4297-aaa1-c0467d387293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:32.952622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ff8ea-25cd-4297-aaa1-c0467d387293 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:32.952908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ff8ea-25cd-4297-aaa1-c0467d387293 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:32.953238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ff8ea-25cd-4297-aaa1-c0467d387293 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:32.965435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ff8ea-25cd-4297-aaa1-c0467d387293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-a66ff8ea-25cd-4297-aaa1-c0467d387293 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:32.974677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:32.975712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:32.981136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 509/1034] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 12:03:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:32.984025 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2b648-01d1-407c-a60d-2c289ef5b31b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:33.105923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2b648-01d1-407c-a60d-2c289ef5b31b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:33.144720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378491cf-edba-417b-b719-131369044be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:33.145652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378491cf-edba-417b-b719-131369044be0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:33.146081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378491cf-edba-417b-b719-131369044be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:33.146611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378491cf-edba-417b-b719-131369044be0 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:33.147552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378491cf-edba-417b-b719-131369044be0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:33.148002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378491cf-edba-417b-b719-131369044be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:33.150785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378491cf-edba-417b-b719-131369044be0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:33.160108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 526/1035] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 12:03:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:33.166134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ff8ea-25cd-4297-aaa1-c0467d387293 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:33.272385 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ff8ea-25cd-4297-aaa1-c0467d387293 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:33.307836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1efad9-50a5-4692-91bf-07a028247206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:33.308728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1efad9-50a5-4692-91bf-07a028247206 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:33.309012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1efad9-50a5-4692-91bf-07a028247206 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:33.309288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1efad9-50a5-4692-91bf-07a028247206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:33.320491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1efad9-50a5-4692-91bf-07a028247206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:33.322607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1efad9-50a5-4692-91bf-07a028247206 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-ba1efad9-50a5-4692-91bf-07a028247206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:33.340209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:33.341176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:33.342048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:33.342048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:33.342048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:33.342048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:33.345918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 510/1036] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:33.347316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378491cf-edba-417b-b719-131369044be0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:33.359861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:33.453891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378491cf-edba-417b-b719-131369044be0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:33.488512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6694e-792a-44d9-91bd-a8e8b0e296ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:33.489176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6694e-792a-44d9-91bd-a8e8b0e296ea None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:33.489455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6694e-792a-44d9-91bd-a8e8b0e296ea None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:33.489727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6694e-792a-44d9-91bd-a8e8b0e296ea None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:33.490465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6694e-792a-44d9-91bd-a8e8b0e296ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:33.490773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6694e-792a-44d9-91bd-a8e8b0e296ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:33.492128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6694e-792a-44d9-91bd-a8e8b0e296ea None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:33.504859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 527/1037] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:33.511182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1efad9-50a5-4692-91bf-07a028247206 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:33.532124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:33.629892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1efad9-50a5-4692-91bf-07a028247206 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:33.665749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefc97a-860d-485b-839f-3d17119c1fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:33.666911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefc97a-860d-485b-839f-3d17119c1fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:33.667200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefc97a-860d-485b-839f-3d17119c1fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:33.667478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefc97a-860d-485b-839f-3d17119c1fe9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:33.675599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefc97a-860d-485b-839f-3d17119c1fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:33.677039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefc97a-860d-485b-839f-3d17119c1fe9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0eefc97a-860d-485b-839f-3d17119c1fe9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:33.691504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:33.692522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:33.698786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 511/1038] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 12:03:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:33.701070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6694e-792a-44d9-91bd-a8e8b0e296ea None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:33.757993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f0bfa-47db-4ece-821f-e6f0dbc7dcc6 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:33.758319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f0bfa-47db-4ece-821f-e6f0dbc7dcc6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:33.759146 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f0bfa-47db-4ece-821f-e6f0dbc7dcc6 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:33.760867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 512/1039] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:33.765702 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb79f-f62e-4364-bdd3-f465239b11e4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:33.765994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb79f-f62e-4364-bdd3-f465239b11e4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:33.766284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3abb79f-f62e-4364-bdd3-f465239b11e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:33.774773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:33.783909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f3abb79f-f62e-4364-bdd3-f465239b11e4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:33.787560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f3abb79f-f62e-4364-bdd3-f465239b11e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:33.812849 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6694e-792a-44d9-91bd-a8e8b0e296ea None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:33.851389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd0c6b-a253-48c4-b8a2-d71ea562496b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:33.852055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd0c6b-a253-48c4-b8a2-d71ea562496b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:33.852365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd0c6b-a253-48c4-b8a2-d71ea562496b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:33.852639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd0c6b-a253-48c4-b8a2-d71ea562496b None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:33.853476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd0c6b-a253-48c4-b8a2-d71ea562496b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:33.853785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd0c6b-a253-48c4-b8a2-d71ea562496b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:33.855240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd0c6b-a253-48c4-b8a2-d71ea562496b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:33.859026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 513/1040] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:33.884108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 528/1041] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 12:03:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:33.884279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb79f-f62e-4364-bdd3-f465239b11e4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:33.886969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd0c6b-a253-48c4-b8a2-d71ea562496b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:33.893366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:33.988195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3abb79f-f62e-4364-bdd3-f465239b11e4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:33.995903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd0c6b-a253-48c4-b8a2-d71ea562496b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:34.026863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:34.030711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d92bbe-0b47-46c9-8741-e9fa53b1f548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:34.031585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d92bbe-0b47-46c9-8741-e9fa53b1f548 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:34.031874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d92bbe-0b47-46c9-8741-e9fa53b1f548 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:34.032151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d92bbe-0b47-46c9-8741-e9fa53b1f548 None admin] PATH_INFO: `/v3/services/image` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:34.032819 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d92bbe-0b47-46c9-8741-e9fa53b1f548 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:34.034292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d92bbe-0b47-46c9-8741-e9fa53b1f548 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:34.036440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51085407-f3ff-47ae-a43b-27ec51ad88db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:34.042393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51085407-f3ff-47ae-a43b-27ec51ad88db None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:34.042393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51085407-f3ff-47ae-a43b-27ec51ad88db None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:34.042393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51085407-f3ff-47ae-a43b-27ec51ad88db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-b0d92bbe-0b47-46c9-8741-e9fa53b1f548 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:34.047331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:34.048283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 12:03:34.049216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:34.053730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 514/1042] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 19 12:03:33 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:34.054362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d92bbe-0b47-46c9-8741-e9fa53b1f548 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:34.112127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51085407-f3ff-47ae-a43b-27ec51ad88db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:34.113575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51085407-f3ff-47ae-a43b-27ec51ad88db None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:34.134008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:34.167806 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d92bbe-0b47-46c9-8741-e9fa53b1f548 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:34.211094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a32ac-17a0-4a0a-9334-4a733b74c501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:34.212469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a32ac-17a0-4a0a-9334-4a733b74c501 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:34.212766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a32ac-17a0-4a0a-9334-4a733b74c501 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:34.213171 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a32ac-17a0-4a0a-9334-4a733b74c501 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:34.213848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a32ac-17a0-4a0a-9334-4a733b74c501 None admin] RBAC: Adding query filter params (name=image) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:34.214164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a32ac-17a0-4a0a-9334-4a733b74c501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:34.215734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a32ac-17a0-4a0a-9334-4a733b74c501 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:34.224252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 529/1043] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:33 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 12:03:34.224428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 515/1044] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:34.225973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51085407-f3ff-47ae-a43b-27ec51ad88db None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:34.226082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a32ac-17a0-4a0a-9334-4a733b74c501 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:34.257230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:34.335450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a32ac-17a0-4a0a-9334-4a733b74c501 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:34.338041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51085407-f3ff-47ae-a43b-27ec51ad88db None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:34.382064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2de9c-e6af-4f4f-9607-96fa73770575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:34.382757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2de9c-e6af-4f4f-9607-96fa73770575 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:34.383044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2de9c-e6af-4f4f-9607-96fa73770575 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:34.383318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2de9c-e6af-4f4f-9607-96fa73770575 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:34.384244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2de9c-e6af-4f4f-9607-96fa73770575 None admin] RBAC: Adding query filter params (type=image) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:34.384435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0b8af-c235-4d23-af77-9810af62a582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:34.384875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2de9c-e6af-4f4f-9607-96fa73770575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:34.385198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0b8af-c235-4d23-af77-9810af62a582 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:34.385480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0b8af-c235-4d23-af77-9810af62a582 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:34.385752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0b8af-c235-4d23-af77-9810af62a582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:34.385913 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2de9c-e6af-4f4f-9607-96fa73770575 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:34.388613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:34.391990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 516/1045] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:34.397275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2de9c-e6af-4f4f-9607-96fa73770575 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:34.399904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0b8af-c235-4d23-af77-9810af62a582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:34.402587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0b8af-c235-4d23-af77-9810af62a582 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-8af0b8af-c235-4d23-af77-9810af62a582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:34.427700 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:34.428628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:34.429508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:34.429508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:34.429508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:34.429508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:34.429508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 530/1046] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:34.435289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0b8af-c235-4d23-af77-9810af62a582 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:34.507243 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2de9c-e6af-4f4f-9607-96fa73770575 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:34.520845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:34.544569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0b8af-c235-4d23-af77-9810af62a582 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:34.544808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f50637-b015-4d3d-a084-8c391a84735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:34.545409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f50637-b015-4d3d-a084-8c391a84735b None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:34.545698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f50637-b015-4d3d-a084-8c391a84735b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:34.546028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f50637-b015-4d3d-a084-8c391a84735b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:34.546851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f50637-b015-4d3d-a084-8c391a84735b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:34.548248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f50637-b015-4d3d-a084-8c391a84735b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:34.581386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed218d-bba4-4bf7-a8ff-ee1b877bd2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:34.582064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed218d-bba4-4bf7-a8ff-ee1b877bd2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:34.582328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed218d-bba4-4bf7-a8ff-ee1b877bd2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:34.582641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed218d-bba4-4bf7-a8ff-ee1b877bd2b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:34.583355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed218d-bba4-4bf7-a8ff-ee1b877bd2b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:34.583669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed218d-bba4-4bf7-a8ff-ee1b877bd2b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:34.585026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed218d-bba4-4bf7-a8ff-ee1b877bd2b7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:34.597746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 531/1047] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:34.604019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed218d-bba4-4bf7-a8ff-ee1b877bd2b7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:34.641229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 517/1048] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 12:03:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:03:34.727642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed218d-bba4-4bf7-a8ff-ee1b877bd2b7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:34.777381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6b518-a7fc-4a39-bb34-1ba5e0b72a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:34.778192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6b518-a7fc-4a39-bb34-1ba5e0b72a55 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:34.778503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6b518-a7fc-4a39-bb34-1ba5e0b72a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:34.778778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6b518-a7fc-4a39-bb34-1ba5e0b72a55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:34.781041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:34.787454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6b518-a7fc-4a39-bb34-1ba5e0b72a55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:34.788931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6b518-a7fc-4a39-bb34-1ba5e0b72a55 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-dee6b518-a7fc-4a39-bb34-1ba5e0b72a55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:34.807146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:34.808118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:34.809053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:34.809053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:34.809053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:34.809053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:34.809053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:34.809053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:34.809053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:34.809053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:34.809053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 532/1049] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:34.815844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f50637-b015-4d3d-a084-8c391a84735b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:34.885161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:34.934661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f50637-b015-4d3d-a084-8c391a84735b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:34.968839 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf8b4b-3c4d-434b-8f78-a4d65cd4499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:34.969504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf8b4b-3c4d-434b-8f78-a4d65cd4499f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:34.969787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf8b4b-3c4d-434b-8f78-a4d65cd4499f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:34.970063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf8b4b-3c4d-434b-8f78-a4d65cd4499f None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:34.970895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf8b4b-3c4d-434b-8f78-a4d65cd4499f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:34.971211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf8b4b-3c4d-434b-8f78-a4d65cd4499f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:34.972640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf8b4b-3c4d-434b-8f78-a4d65cd4499f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:34.987130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 518/1050] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:34.993421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6b518-a7fc-4a39-bb34-1ba5e0b72a55 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:35.113433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6b518-a7fc-4a39-bb34-1ba5e0b72a55 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:35.149187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee1226-3d41-41ec-8741-ede94b8e60ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:35.149871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee1226-3d41-41ec-8741-ede94b8e60ff None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:35.150154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee1226-3d41-41ec-8741-ede94b8e60ff None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:35.150450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee1226-3d41-41ec-8741-ede94b8e60ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:35.151281 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee1226-3d41-41ec-8741-ede94b8e60ff None admin] RBAC: Adding query filter params (role.id=d538a3ae543f4c1897c8c3d9c8fb8066, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=bb6776311f354b748d563c4f1d793972) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:35.151600 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee1226-3d41-41ec-8741-ede94b8e60ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:35.153061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee1226-3d41-41ec-8741-ede94b8e60ff None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:35.166432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 533/1051] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:34 2022] GET /identity/v3/role_assignments?user.id=bb6776311f354b748d563c4f1d793972&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=d538a3ae543f4c1897c8c3d9c8fb8066 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:35.222620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515a3b8-7a8d-4deb-9c3e-9810a2fd4fcb None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:35.222928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515a3b8-7a8d-4deb-9c3e-9810a2fd4fcb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:35.223225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515a3b8-7a8d-4deb-9c3e-9810a2fd4fcb None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:35.224709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 519/1052] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:35.231245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd2689-821d-4db2-8739-afa522457f8c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:35.231641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd2689-821d-4db2-8739-afa522457f8c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:35.231822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd2689-821d-4db2-8739-afa522457f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:35.250183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c8cd2689-821d-4db2-8739-afa522457f8c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:35.251490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c8cd2689-821d-4db2-8739-afa522457f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:35.315501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:35.331261 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 534/1053] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:35.355714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515a3b8-7a8d-4deb-9c3e-9810a2fd4fcb None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:35.459058 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515a3b8-7a8d-4deb-9c3e-9810a2fd4fcb None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:35.478579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:35.495998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fadde3-ac4e-4a0d-ba09-d2ddaae8cd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:35.496984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fadde3-ac4e-4a0d-ba09-d2ddaae8cd95 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:35.497120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fadde3-ac4e-4a0d-ba09-d2ddaae8cd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:35.497399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fadde3-ac4e-4a0d-ba09-d2ddaae8cd95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:35.507852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fadde3-ac4e-4a0d-ba09-d2ddaae8cd95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-a2fadde3-ac4e-4a0d-ba09-d2ddaae8cd95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:35.516186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:35.517408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:35.518476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 520/1054] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:03:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:35.525361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd2689-821d-4db2-8739-afa522457f8c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:35.630611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd2689-821d-4db2-8739-afa522457f8c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:35.665744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b6170-8fdc-411e-8c39-d46ad70f60ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:35.666714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b6170-8fdc-411e-8c39-d46ad70f60ea None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:35.667003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b6170-8fdc-411e-8c39-d46ad70f60ea None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:35.667280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b6170-8fdc-411e-8c39-d46ad70f60ea None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:35.668019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b6170-8fdc-411e-8c39-d46ad70f60ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:35.668365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b6170-8fdc-411e-8c39-d46ad70f60ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:35.669739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b6170-8fdc-411e-8c39-d46ad70f60ea None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:35.680213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 535/1055] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:03:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:35.686858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fadde3-ac4e-4a0d-ba09-d2ddaae8cd95 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:35.791935 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fadde3-ac4e-4a0d-ba09-d2ddaae8cd95 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:35.825191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766792f-9722-41b5-ac52-bea2ed3e32de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:35.826113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766792f-9722-41b5-ac52-bea2ed3e32de None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:35.826399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766792f-9722-41b5-ac52-bea2ed3e32de None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:35.826748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766792f-9722-41b5-ac52-bea2ed3e32de None admin] PATH_INFO: `/v3/users/d19ec9e0a9034a7a9267c42eaee6503c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:35.841219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766792f-9722-41b5-ac52-bea2ed3e32de None admin] RBAC: Authorizing `identity:get_user(user_id=d19ec9e0a9034a7a9267c42eaee6503c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:35.842691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766792f-9722-41b5-ac52-bea2ed3e32de None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:35.846645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 521/1056] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:03:35 2022] GET /identity/v3/users/d19ec9e0a9034a7a9267c42eaee6503c => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:35.850096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b6170-8fdc-411e-8c39-d46ad70f60ea None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:35.906888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:35.958751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b6170-8fdc-411e-8c39-d46ad70f60ea None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:35.993255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5b860-29e5-4b22-b486-2464dd7c4489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:35.993908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5b860-29e5-4b22-b486-2464dd7c4489 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:35.994190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5b860-29e5-4b22-b486-2464dd7c4489 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:35.994490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5b860-29e5-4b22-b486-2464dd7c4489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:36.073173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5b860-29e5-4b22-b486-2464dd7c4489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:36.074582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5b860-29e5-4b22-b486-2464dd7c4489 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:36.122703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:36.174584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 536/1057] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:03:36.178309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766792f-9722-41b5-ac52-bea2ed3e32de None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:36.213001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:36.280287 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766792f-9722-41b5-ac52-bea2ed3e32de None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:36.318380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e882f4-90f0-4cb9-b89f-63d54cf3000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:36.319352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e882f4-90f0-4cb9-b89f-63d54cf3000b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:36.319673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e882f4-90f0-4cb9-b89f-63d54cf3000b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:36.319851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e882f4-90f0-4cb9-b89f-63d54cf3000b None admin] PATH_INFO: `/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:36.321075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e882f4-90f0-4cb9-b89f-63d54cf3000b None admin] RBAC: Authorizing `identity:get_project(project_id=3d2f600e49bc47e2a65ffb8fcda9d1fc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:36.322652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e882f4-90f0-4cb9-b89f-63d54cf3000b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:36.329278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 522/1058] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:03:36 2022] GET /identity/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:36.332386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5b860-29e5-4b22-b486-2464dd7c4489 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:36.430156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5b860-29e5-4b22-b486-2464dd7c4489 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:36.464346 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6df4c-0714-4ac5-8206-56b2a191a12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:36.464998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6df4c-0714-4ac5-8206-56b2a191a12b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:36.465280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6df4c-0714-4ac5-8206-56b2a191a12b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:36.465555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6df4c-0714-4ac5-8206-56b2a191a12b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:36.466398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6df4c-0714-4ac5-8206-56b2a191a12b None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=3d2f600e49bc47e2a65ffb8fcda9d1fc, user.id=d19ec9e0a9034a7a9267c42eaee6503c) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:36.466736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6df4c-0714-4ac5-8206-56b2a191a12b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:36.468160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6df4c-0714-4ac5-8206-56b2a191a12b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:36.482684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 537/1059] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:36 2022] GET /identity/v3/role_assignments?user.id=d19ec9e0a9034a7a9267c42eaee6503c&scope.project.id=3d2f600e49bc47e2a65ffb8fcda9d1fc&role.id=bd295cb5d1194b469beff0381c6b23af => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:37.029396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ba88-53b9-485b-889d-b16afc3a1cf2 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:37.029853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ba88-53b9-485b-889d-b16afc3a1cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:37.029942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ba88-53b9-485b-889d-b16afc3a1cf2 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:37.031379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 523/1060] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:37.037063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d728-7205-4ee7-8348-b581a93af8d6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:37.037519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d728-7205-4ee7-8348-b581a93af8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:37.037726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d728-7205-4ee7-8348-b581a93af8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:37.055801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9770d728-7205-4ee7-8348-b581a93af8d6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:37.057052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9770d728-7205-4ee7-8348-b581a93af8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:37.066932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:37.126143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 538/1061] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:37.148152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268ba88-53b9-485b-889d-b16afc3a1cf2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:37.266647 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268ba88-53b9-485b-889d-b16afc3a1cf2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:37.268501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:37.313373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3a487-0485-4409-b331-e3b56eec7764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:37.314320 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3a487-0485-4409-b331-e3b56eec7764 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:37.314731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3a487-0485-4409-b331-e3b56eec7764 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:37.315163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3a487-0485-4409-b331-e3b56eec7764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:37.399641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3a487-0485-4409-b331-e3b56eec7764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:37.401101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3a487-0485-4409-b331-e3b56eec7764 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:37.432174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb6357-743c-4f02-891c-2a4b04b3fc11 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:37.432584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb6357-743c-4f02-891c-2a4b04b3fc11 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:37.432747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb6357-743c-4f02-891c-2a4b04b3fc11 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:37.434120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 539/1062] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:37.439482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b503-86f7-4acd-b011-9c09b6548cf7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:37.439851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b503-86f7-4acd-b011-9c09b6548cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:37.440231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b503-86f7-4acd-b011-9c09b6548cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:37.457285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8df8b503-86f7-4acd-b011-9c09b6548cf7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:37.458627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8df8b503-86f7-4acd-b011-9c09b6548cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:37.509342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 524/1063] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:03:37.515261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3a487-0485-4409-b331-e3b56eec7764 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:37.517237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:37.529615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 540/1064] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:37.553640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b503-86f7-4acd-b011-9c09b6548cf7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:37.625329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3a487-0485-4409-b331-e3b56eec7764 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:37.636305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:37.665462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b503-86f7-4acd-b011-9c09b6548cf7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:37.667499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7700f5-1ea5-4fc5-a8d2-ae0b1611833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:37.668825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7700f5-1ea5-4fc5-a8d2-ae0b1611833f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:37.669259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7700f5-1ea5-4fc5-a8d2-ae0b1611833f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:37.669705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7700f5-1ea5-4fc5-a8d2-ae0b1611833f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:37.681043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7700f5-1ea5-4fc5-a8d2-ae0b1611833f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:37.682534 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7700f5-1ea5-4fc5-a8d2-ae0b1611833f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6d7700f5-1ea5-4fc5-a8d2-ae0b1611833f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:37.702304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:37.703555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:37.704465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:37.704465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:37.704465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:37.704465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:37.704465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 525/1065] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:37.708698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16694f-da90-4f91-b091-88c4f08852e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:37.709577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16694f-da90-4f91-b091-88c4f08852e9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:37.709779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7700f5-1ea5-4fc5-a8d2-ae0b1611833f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:37.709883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16694f-da90-4f91-b091-88c4f08852e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:37.710183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16694f-da90-4f91-b091-88c4f08852e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:37.722383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16694f-da90-4f91-b091-88c4f08852e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-1b16694f-da90-4f91-b091-88c4f08852e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:37.731764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:37.732712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:37.733712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 541/1066] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 12:03:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:37.738910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16694f-da90-4f91-b091-88c4f08852e9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:37.771637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:37.822151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7700f5-1ea5-4fc5-a8d2-ae0b1611833f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:37.849895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16694f-da90-4f91-b091-88c4f08852e9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:37.859919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20528d9b-48af-43bb-8364-13cc4a8fd4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:37.860898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528d9b-48af-43bb-8364-13cc4a8fd4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:37.861190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528d9b-48af-43bb-8364-13cc4a8fd4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:37.861473 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528d9b-48af-43bb-8364-13cc4a8fd4a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:37.862197 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528d9b-48af-43bb-8364-13cc4a8fd4a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:37.862539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528d9b-48af-43bb-8364-13cc4a8fd4a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:37.863902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528d9b-48af-43bb-8364-13cc4a8fd4a4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:37.877224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 526/1067] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:37.885414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b4435-dd11-4b75-942a-ab0e0e9fa57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:37.886105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b4435-dd11-4b75-942a-ab0e0e9fa57f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:37.886389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b4435-dd11-4b75-942a-ab0e0e9fa57f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:37.886698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b4435-dd11-4b75-942a-ab0e0e9fa57f None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:37.887458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b4435-dd11-4b75-942a-ab0e0e9fa57f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:37.887770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b4435-dd11-4b75-942a-ab0e0e9fa57f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:37.889228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b4435-dd11-4b75-942a-ab0e0e9fa57f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:37.891175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:37.900115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 542/1068] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 12:03:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:37.905993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20528d9b-48af-43bb-8364-13cc4a8fd4a4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:38.020720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20528d9b-48af-43bb-8364-13cc4a8fd4a4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:38.042291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:38.064418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ffc68-8322-4591-960f-476f95c801e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:38.065216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ffc68-8322-4591-960f-476f95c801e6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:38.065501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ffc68-8322-4591-960f-476f95c801e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:38.065875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ffc68-8322-4591-960f-476f95c801e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:38.079651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ffc68-8322-4591-960f-476f95c801e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:38.081106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ffc68-8322-4591-960f-476f95c801e6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-376ffc68-8322-4591-960f-476f95c801e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:38.100078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:38.101031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:38.102670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:38.102670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:38.102670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:38.102670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:38.102670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 527/1069] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:38.107759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b4435-dd11-4b75-942a-ab0e0e9fa57f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:38.216402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b4435-dd11-4b75-942a-ab0e0e9fa57f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:38.252515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea661e1-4ae3-49c6-8db9-705f882f3c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:38.253182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea661e1-4ae3-49c6-8db9-705f882f3c88 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:38.253463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea661e1-4ae3-49c6-8db9-705f882f3c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:38.253738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea661e1-4ae3-49c6-8db9-705f882f3c88 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:38.254473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea661e1-4ae3-49c6-8db9-705f882f3c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:38.254785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea661e1-4ae3-49c6-8db9-705f882f3c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:38.256144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea661e1-4ae3-49c6-8db9-705f882f3c88 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:38.267723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 543/1070] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:38.273692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ffc68-8322-4591-960f-476f95c801e6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:38.377365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ffc68-8322-4591-960f-476f95c801e6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:38.411000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e55abb-47c6-4aca-9d6c-6d47308b2eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:38.411785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e55abb-47c6-4aca-9d6c-6d47308b2eec None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:38.412074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e55abb-47c6-4aca-9d6c-6d47308b2eec None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:38.412353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e55abb-47c6-4aca-9d6c-6d47308b2eec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:38.421066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e55abb-47c6-4aca-9d6c-6d47308b2eec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:38.422560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e55abb-47c6-4aca-9d6c-6d47308b2eec None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-93e55abb-47c6-4aca-9d6c-6d47308b2eec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:38.437632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:38.438700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:38.439653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 528/1071] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 12:03:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:38.444761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea661e1-4ae3-49c6-8db9-705f882f3c88 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:38.502231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:38.550747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea661e1-4ae3-49c6-8db9-705f882f3c88 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:38.587803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c6cc3-085b-4d7a-aca0-67bf21d18cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:38.588511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c6cc3-085b-4d7a-aca0-67bf21d18cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:38.588807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c6cc3-085b-4d7a-aca0-67bf21d18cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:38.589084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c6cc3-085b-4d7a-aca0-67bf21d18cf2 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:38.589923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c6cc3-085b-4d7a-aca0-67bf21d18cf2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:38.590239 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c6cc3-085b-4d7a-aca0-67bf21d18cf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:38.591725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c6cc3-085b-4d7a-aca0-67bf21d18cf2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:38.609872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 544/1072] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 12:03:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:38.615636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e55abb-47c6-4aca-9d6c-6d47308b2eec None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:38.644770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:38.722521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e55abb-47c6-4aca-9d6c-6d47308b2eec None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:38.729375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c55a65-070e-4cc1-9498-fb2256b6d60f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:38.729671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c55a65-070e-4cc1-9498-fb2256b6d60f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:38.729954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c55a65-070e-4cc1-9498-fb2256b6d60f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:38.731989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 545/1073] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:38.737548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d3d7-3e3b-408a-a4ef-64343cef6a54 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:38.737843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d3d7-3e3b-408a-a4ef-64343cef6a54 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:38.738123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d3d7-3e3b-408a-a4ef-64343cef6a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:38.755679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ae23d3d7-3e3b-408a-a4ef-64343cef6a54 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:38.756796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ae23d3d7-3e3b-408a-a4ef-64343cef6a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:38.757884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d5732-f552-454a-8acb-b02d44f98f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:38.759440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d5732-f552-454a-8acb-b02d44f98f6f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:38.759714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d5732-f552-454a-8acb-b02d44f98f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:38.759992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d5732-f552-454a-8acb-b02d44f98f6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:38.771406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d5732-f552-454a-8acb-b02d44f98f6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:38.772840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d5732-f552-454a-8acb-b02d44f98f6f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-769d5732-f552-454a-8acb-b02d44f98f6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:38.790654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:38.791911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:38.792812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:38.792812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:38.792812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:38.792812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:38.792812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 529/1074] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:38.797954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d5732-f552-454a-8acb-b02d44f98f6f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:38.828603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 546/1075] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:38.851529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23d3d7-3e3b-408a-a4ef-64343cef6a54 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:38.882130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:38.902104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:38.903963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d5732-f552-454a-8acb-b02d44f98f6f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:38.939030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299bf81-a416-45ba-a5c3-3cb4c596eb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:38.939692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299bf81-a416-45ba-a5c3-3cb4c596eb58 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:38.939979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299bf81-a416-45ba-a5c3-3cb4c596eb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:38.940306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299bf81-a416-45ba-a5c3-3cb4c596eb58 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:38.941029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299bf81-a416-45ba-a5c3-3cb4c596eb58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:38.941336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299bf81-a416-45ba-a5c3-3cb4c596eb58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:38.942719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299bf81-a416-45ba-a5c3-3cb4c596eb58 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:38.955307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 530/1076] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:38.957918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23d3d7-3e3b-408a-a4ef-64343cef6a54 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:38.961054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299bf81-a416-45ba-a5c3-3cb4c596eb58 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:38.995298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb099ea8-5262-4e62-9b63-7fde891d2e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:38.996115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb099ea8-5262-4e62-9b63-7fde891d2e64 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:38.996393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb099ea8-5262-4e62-9b63-7fde891d2e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:38.996662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb099ea8-5262-4e62-9b63-7fde891d2e64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:39.010818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb099ea8-5262-4e62-9b63-7fde891d2e64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-eb099ea8-5262-4e62-9b63-7fde891d2e64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:39.020737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:39.022852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:39.024586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 547/1077] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:03:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:39.027466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb099ea8-5262-4e62-9b63-7fde891d2e64 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:39.072099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299bf81-a416-45ba-a5c3-3cb4c596eb58 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:39.107018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ee58d-b2d2-4817-a2c3-dfc72e1731d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:39.107802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ee58d-b2d2-4817-a2c3-dfc72e1731d3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:39.108086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ee58d-b2d2-4817-a2c3-dfc72e1731d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:39.108361 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ee58d-b2d2-4817-a2c3-dfc72e1731d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:39.115026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ee58d-b2d2-4817-a2c3-dfc72e1731d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:39.116449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ee58d-b2d2-4817-a2c3-dfc72e1731d3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:39.118383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-a69ee58d-b2d2-4817-a2c3-dfc72e1731d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:39.131077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:39.132166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:39.133792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:39.133792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:39.133792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:39.133792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:39.133792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:39.133792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:39.133792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:39.133792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:39.133792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 531/1078] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:39.135133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb099ea8-5262-4e62-9b63-7fde891d2e64 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:39.137836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:39.138465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ee58d-b2d2-4817-a2c3-dfc72e1731d3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:39.170022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c783b-7797-44b1-8212-562f38549bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:39.170724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c783b-7797-44b1-8212-562f38549bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:39.171022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c783b-7797-44b1-8212-562f38549bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:39.171304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c783b-7797-44b1-8212-562f38549bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:39.172056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c783b-7797-44b1-8212-562f38549bb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:39.172361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c783b-7797-44b1-8212-562f38549bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:39.173728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c783b-7797-44b1-8212-562f38549bb2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:39.182337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 548/1079] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:03:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:39.188092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c783b-7797-44b1-8212-562f38549bb2 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:39.240138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ee58d-b2d2-4817-a2c3-dfc72e1731d3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:39.274597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb74f6-6403-4794-89f5-82f93e330315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:39.275190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb74f6-6403-4794-89f5-82f93e330315 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:39.275478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb74f6-6403-4794-89f5-82f93e330315 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:39.275752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb74f6-6403-4794-89f5-82f93e330315 None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:39.276536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb74f6-6403-4794-89f5-82f93e330315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:39.276847 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb74f6-6403-4794-89f5-82f93e330315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:39.278272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb74f6-6403-4794-89f5-82f93e330315 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:39.286661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c783b-7797-44b1-8212-562f38549bb2 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:39.290697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 532/1080] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:39.296783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb74f6-6403-4794-89f5-82f93e330315 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:39.322471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec0f93-2c4e-42e8-a449-ea17883d46b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:39.323346 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec0f93-2c4e-42e8-a449-ea17883d46b7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:39.323631 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec0f93-2c4e-42e8-a449-ea17883d46b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:39.323906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec0f93-2c4e-42e8-a449-ea17883d46b7 None admin] PATH_INFO: `/v3/users/d19ec9e0a9034a7a9267c42eaee6503c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:39.325100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec0f93-2c4e-42e8-a449-ea17883d46b7 None admin] RBAC: Authorizing `identity:get_user(user_id=d19ec9e0a9034a7a9267c42eaee6503c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:39.326563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec0f93-2c4e-42e8-a449-ea17883d46b7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:39.328190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 549/1081] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:03:39 2022] GET /identity/v3/users/d19ec9e0a9034a7a9267c42eaee6503c => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:39.333660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec0f93-2c4e-42e8-a449-ea17883d46b7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:39.358906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:39.376692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:39.403402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb74f6-6403-4794-89f5-82f93e330315 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:39.440841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff206c-a272-41c2-888d-356a54ac73f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:39.441729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff206c-a272-41c2-888d-356a54ac73f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:39.442022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff206c-a272-41c2-888d-356a54ac73f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:39.442298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff206c-a272-41c2-888d-356a54ac73f5 None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/bb6776311f354b748d563c4f1d793972/roles/d538a3ae543f4c1897c8c3d9c8fb8066` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:39.444556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff206c-a272-41c2-888d-356a54ac73f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=bb6776311f354b748d563c4f1d793972, role_id=d538a3ae543f4c1897c8c3d9c8fb8066)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:39.444665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec0f93-2c4e-42e8-a449-ea17883d46b7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:39.446070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff206c-a272-41c2-888d-356a54ac73f5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:39.478430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc736e-9ab3-482f-b589-003a52e95d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:39.479324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc736e-9ab3-482f-b589-003a52e95d1e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:39.479610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc736e-9ab3-482f-b589-003a52e95d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:39.479948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc736e-9ab3-482f-b589-003a52e95d1e None admin] PATH_INFO: `/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:39.481242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc736e-9ab3-482f-b589-003a52e95d1e None admin] RBAC: Authorizing `identity:get_project(project_id=3d2f600e49bc47e2a65ffb8fcda9d1fc)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:39.482995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc736e-9ab3-482f-b589-003a52e95d1e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:39.485265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 550/1082] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:03:39 2022] GET /identity/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:39.491096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc736e-9ab3-482f-b589-003a52e95d1e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:39.590274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc736e-9ab3-482f-b589-003a52e95d1e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:39.591743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:39.619689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 533/1083] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:03:39 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/bb6776311f354b748d563c4f1d793972/roles/d538a3ae543f4c1897c8c3d9c8fb8066 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:03:39.628839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05aa0b6-9e08-43ce-a8ab-c793593a8486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:39.628839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05aa0b6-9e08-43ce-a8ab-c793593a8486 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:39.628839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05aa0b6-9e08-43ce-a8ab-c793593a8486 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:39.629179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05aa0b6-9e08-43ce-a8ab-c793593a8486 None admin] PATH_INFO: `/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc/users/d19ec9e0a9034a7a9267c42eaee6503c/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:39.631095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05aa0b6-9e08-43ce-a8ab-c793593a8486 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d2f600e49bc47e2a65ffb8fcda9d1fc, user_id=d19ec9e0a9034a7a9267c42eaee6503c, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:39.632565 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05aa0b6-9e08-43ce-a8ab-c793593a8486 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:39.854031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 551/1084] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:03:39 2022] PUT /identity/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc/users/d19ec9e0a9034a7a9267c42eaee6503c/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:03:42.064483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3f947-c79f-4e9b-9551-e492bb8eff96 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.065004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3f947-c79f-4e9b-9551-e492bb8eff96 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.065122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3f947-c79f-4e9b-9551-e492bb8eff96 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.066425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 534/1085] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:42.072543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc516364-9475-421d-abc8-3d7f3527c18e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.072987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc516364-9475-421d-abc8-3d7f3527c18e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.073117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc516364-9475-421d-abc8-3d7f3527c18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.089820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cc516364-9475-421d-abc8-3d7f3527c18e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:42.091104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cc516364-9475-421d-abc8-3d7f3527c18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:42.108375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ffa9-38bb-4f80-ae9f-620b228273b3 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.108672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ffa9-38bb-4f80-ae9f-620b228273b3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.108959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8ffa9-38bb-4f80-ae9f-620b228273b3 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.110310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 535/1086] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:03:42.115166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262755ef-4c9e-4926-8663-cb597a014edd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.115663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262755ef-4c9e-4926-8663-cb597a014edd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.115944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262755ef-4c9e-4926-8663-cb597a014edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.133312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-262755ef-4c9e-4926-8663-cb597a014edd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:42.134531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-262755ef-4c9e-4926-8663-cb597a014edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:42.159796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 552/1087] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:42.184653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc516364-9475-421d-abc8-3d7f3527c18e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:42.204873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 536/1088] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:03:42.228345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262755ef-4c9e-4926-8663-cb597a014edd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:42.276098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:42.277414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:42.284303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc516364-9475-421d-abc8-3d7f3527c18e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:42.318826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee7c11-a6d5-448f-8976-69c164680d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:42.320265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee7c11-a6d5-448f-8976-69c164680d7e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.320805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee7c11-a6d5-448f-8976-69c164680d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.324875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee7c11-a6d5-448f-8976-69c164680d7e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.328229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262755ef-4c9e-4926-8663-cb597a014edd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:42.333407 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee7c11-a6d5-448f-8976-69c164680d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-afee7c11-a6d5-448f-8976-69c164680d7e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:42.345177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:42.346144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:42.353039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 553/1089] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 19 12:03:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:42.354335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee7c11-a6d5-448f-8976-69c164680d7e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:42.362985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7174b-d506-4e47-89aa-5ab75fa6c36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:42.363786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7174b-d506-4e47-89aa-5ab75fa6c36b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.364071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7174b-d506-4e47-89aa-5ab75fa6c36b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.364348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7174b-d506-4e47-89aa-5ab75fa6c36b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.375143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7174b-d506-4e47-89aa-5ab75fa6c36b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-aaf7174b-d506-4e47-89aa-5ab75fa6c36b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:03:42.382789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:42.389574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:42.390751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 537/1090] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:03:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:42.391708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7174b-d506-4e47-89aa-5ab75fa6c36b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:42.459984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee7c11-a6d5-448f-8976-69c164680d7e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:42.492509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7174b-d506-4e47-89aa-5ab75fa6c36b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:42.494167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb036d-1e9a-418e-a602-364ff8c58555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:42.494865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb036d-1e9a-418e-a602-364ff8c58555 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.495147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb036d-1e9a-418e-a602-364ff8c58555 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.495421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb036d-1e9a-418e-a602-364ff8c58555 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.496174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb036d-1e9a-418e-a602-364ff8c58555 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:42.496465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb036d-1e9a-418e-a602-364ff8c58555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:42.497820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb036d-1e9a-418e-a602-364ff8c58555 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:42.501608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:42.505853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 554/1091] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 12:03:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:42.511407 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb036d-1e9a-418e-a602-364ff8c58555 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:42.526684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862470-4c42-4053-82f4-2aa3782f95d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:42.527282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862470-4c42-4053-82f4-2aa3782f95d6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.527572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862470-4c42-4053-82f4-2aa3782f95d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.527686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:42.527849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862470-4c42-4053-82f4-2aa3782f95d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.528609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862470-4c42-4053-82f4-2aa3782f95d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:42.528997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862470-4c42-4053-82f4-2aa3782f95d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:42.530344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862470-4c42-4053-82f4-2aa3782f95d6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:42.540454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 538/1092] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:03:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:42.546264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862470-4c42-4053-82f4-2aa3782f95d6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:42.615067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb036d-1e9a-418e-a602-364ff8c58555 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:42.647052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862470-4c42-4053-82f4-2aa3782f95d6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:42.649993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c00bd-ab4a-4159-ba1a-93dd1c9731bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:42.650822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c00bd-ab4a-4159-ba1a-93dd1c9731bd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.651112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c00bd-ab4a-4159-ba1a-93dd1c9731bd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.651390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c00bd-ab4a-4159-ba1a-93dd1c9731bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.661364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c00bd-ab4a-4159-ba1a-93dd1c9731bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:42.662855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c00bd-ab4a-4159-ba1a-93dd1c9731bd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-306c00bd-ab4a-4159-ba1a-93dd1c9731bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:42.681547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:42.682725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:42.683650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:42.683650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:42.683650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:42.683650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:42.683876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99951c-cbb7-45c8-9f51-d801e538a0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:42.683976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 555/1093] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:42.684050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99951c-cbb7-45c8-9f51-d801e538a0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.684050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99951c-cbb7-45c8-9f51-d801e538a0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.684222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99951c-cbb7-45c8-9f51-d801e538a0d1 None admin] PATH_INFO: `/v3/users/d19ec9e0a9034a7a9267c42eaee6503c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.685439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99951c-cbb7-45c8-9f51-d801e538a0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=d19ec9e0a9034a7a9267c42eaee6503c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:42.686898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99951c-cbb7-45c8-9f51-d801e538a0d1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:42.688368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c00bd-ab4a-4159-ba1a-93dd1c9731bd None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:42.688798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 539/1094] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:03:42 2022] GET /identity/v3/users/d19ec9e0a9034a7a9267c42eaee6503c => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:42.694142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99951c-cbb7-45c8-9f51-d801e538a0d1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:42.736265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:42.754619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:42.788477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c00bd-ab4a-4159-ba1a-93dd1c9731bd None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:42.795254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99951c-cbb7-45c8-9f51-d801e538a0d1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:42.823001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb4bc4-8fdf-4b62-b175-ce1c239f88ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:42.823658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb4bc4-8fdf-4b62-b175-ce1c239f88ec None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.823943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb4bc4-8fdf-4b62-b175-ce1c239f88ec None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.824219 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb4bc4-8fdf-4b62-b175-ce1c239f88ec None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.824935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb4bc4-8fdf-4b62-b175-ce1c239f88ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:42.825247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb4bc4-8fdf-4b62-b175-ce1c239f88ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:42.826636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb4bc4-8fdf-4b62-b175-ce1c239f88ec None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:42.830129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c49870-73fd-437d-9c79-b0584a8aa1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:42.830811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c49870-73fd-437d-9c79-b0584a8aa1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:42.831132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c49870-73fd-437d-9c79-b0584a8aa1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:42.831414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c49870-73fd-437d-9c79-b0584a8aa1a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:42.839267 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 556/1095] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:42.844770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb4bc4-8fdf-4b62-b175-ce1c239f88ec None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:42.899814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c49870-73fd-437d-9c79-b0584a8aa1a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:42.901254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c49870-73fd-437d-9c79-b0584a8aa1a3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:42.998871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:43.036178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb4bc4-8fdf-4b62-b175-ce1c239f88ec None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:43.062709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:43.067117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 540/1096] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 12:03:43.067998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c49870-73fd-437d-9c79-b0584a8aa1a3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:43.071538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99910761-ec27-421d-85fe-a50aa4c37b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:43.072341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99910761-ec27-421d-85fe-a50aa4c37b12 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:43.072688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99910761-ec27-421d-85fe-a50aa4c37b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:43.073213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99910761-ec27-421d-85fe-a50aa4c37b12 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:43.080857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99910761-ec27-421d-85fe-a50aa4c37b12 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:43.082320 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99910761-ec27-421d-85fe-a50aa4c37b12 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-99910761-ec27-421d-85fe-a50aa4c37b12 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:03:43.097317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:43.098302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:03:43.099635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 557/1097] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 19 12:03:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:03:43.104859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99910761-ec27-421d-85fe-a50aa4c37b12 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:43.175248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c49870-73fd-437d-9c79-b0584a8aa1a3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:43.212225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7feaa-03ac-4171-9249-fe14638e8078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:43.213107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7feaa-03ac-4171-9249-fe14638e8078 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:43.213393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7feaa-03ac-4171-9249-fe14638e8078 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:43.213801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7feaa-03ac-4171-9249-fe14638e8078 None admin] PATH_INFO: `/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:43.214893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7feaa-03ac-4171-9249-fe14638e8078 None admin] RBAC: Authorizing `identity:get_project(project_id=3d2f600e49bc47e2a65ffb8fcda9d1fc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:43.216321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7feaa-03ac-4171-9249-fe14638e8078 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:43.218008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 541/1098] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:03:43 2022] GET /identity/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:43.223948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7feaa-03ac-4171-9249-fe14638e8078 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:43.226395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99910761-ec27-421d-85fe-a50aa4c37b12 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:43.232951 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:43.267341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6170c-ba7c-4f99-a60e-5ba2a55f9ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:43.268016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6170c-ba7c-4f99-a60e-5ba2a55f9ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:43.268305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6170c-ba7c-4f99-a60e-5ba2a55f9ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:43.268588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6170c-ba7c-4f99-a60e-5ba2a55f9ef1 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:43.269418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6170c-ba7c-4f99-a60e-5ba2a55f9ef1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:43.269734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6170c-ba7c-4f99-a60e-5ba2a55f9ef1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:43.271198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6170c-ba7c-4f99-a60e-5ba2a55f9ef1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:43.290118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 558/1099] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 19 12:03:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:43.295869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6170c-ba7c-4f99-a60e-5ba2a55f9ef1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:43.323082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7feaa-03ac-4171-9249-fe14638e8078 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:43.340995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:43.358486 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6a83b-f213-4e8d-ab87-36c68dbacd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:43.359140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6a83b-f213-4e8d-ab87-36c68dbacd68 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:43.359422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6a83b-f213-4e8d-ab87-36c68dbacd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:43.359730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6a83b-f213-4e8d-ab87-36c68dbacd68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:43.360556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6a83b-f213-4e8d-ab87-36c68dbacd68 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=3d2f600e49bc47e2a65ffb8fcda9d1fc, user.id=d19ec9e0a9034a7a9267c42eaee6503c) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:43.360875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6a83b-f213-4e8d-ab87-36c68dbacd68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:43.362321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6a83b-f213-4e8d-ab87-36c68dbacd68 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:43.368671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 542/1100] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:43 2022] GET /identity/v3/role_assignments?user.id=d19ec9e0a9034a7a9267c42eaee6503c&scope.project.id=3d2f600e49bc47e2a65ffb8fcda9d1fc&role.id=bd295cb5d1194b469beff0381c6b23af => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:43.398177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6170c-ba7c-4f99-a60e-5ba2a55f9ef1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:43.437368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ac86e-6def-4ae6-aa50-ebdb7871497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:43.438043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ac86e-6def-4ae6-aa50-ebdb7871497f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:43.438325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ac86e-6def-4ae6-aa50-ebdb7871497f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:43.438634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ac86e-6def-4ae6-aa50-ebdb7871497f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:43.514103 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ac86e-6def-4ae6-aa50-ebdb7871497f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:43.515627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ac86e-6def-4ae6-aa50-ebdb7871497f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:43.568423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:43.625655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 559/1101] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 12:03:43.631424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6a83b-f213-4e8d-ab87-36c68dbacd68 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:43.701591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:43.732608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6a83b-f213-4e8d-ab87-36c68dbacd68 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:43.767320 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b769500-500d-4a11-b33d-ed7e9119e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:43.768111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b769500-500d-4a11-b33d-ed7e9119e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:43.768394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b769500-500d-4a11-b33d-ed7e9119e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:43.768708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b769500-500d-4a11-b33d-ed7e9119e6c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:43.779531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b769500-500d-4a11-b33d-ed7e9119e6c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:43.781005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b769500-500d-4a11-b33d-ed7e9119e6c6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-0b769500-500d-4a11-b33d-ed7e9119e6c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:03:43.798579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:43.799606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:43.800562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:03:43.800562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:03:43.800562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:03:43.800562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:43.802551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 543/1102] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:43.805822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ac86e-6def-4ae6-aa50-ebdb7871497f None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:43.902967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ac86e-6def-4ae6-aa50-ebdb7871497f None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:43.937637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d491e-70eb-4cd7-b39a-e19974c5fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:43.938300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d491e-70eb-4cd7-b39a-e19974c5fe81 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:43.938616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d491e-70eb-4cd7-b39a-e19974c5fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:43.938893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d491e-70eb-4cd7-b39a-e19974c5fe81 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:43.939608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d491e-70eb-4cd7-b39a-e19974c5fe81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:43.939915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d491e-70eb-4cd7-b39a-e19974c5fe81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:43.941293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d491e-70eb-4cd7-b39a-e19974c5fe81 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:43.953463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 560/1103] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:43.958828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b769500-500d-4a11-b33d-ed7e9119e6c6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:44.061363 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b769500-500d-4a11-b33d-ed7e9119e6c6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:44.085044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:44.096776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249baa8-ad0b-4f56-8df5-e331c4b95659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:44.097565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249baa8-ad0b-4f56-8df5-e331c4b95659 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:44.097846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249baa8-ad0b-4f56-8df5-e331c4b95659 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:44.098120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249baa8-ad0b-4f56-8df5-e331c4b95659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:44.104795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249baa8-ad0b-4f56-8df5-e331c4b95659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:44.106224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249baa8-ad0b-4f56-8df5-e331c4b95659 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-6249baa8-ad0b-4f56-8df5-e331c4b95659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:03:44.121447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:03:44.122379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:03:44.128291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:03:44.128291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:03:44.128291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:03:44.128291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:03:44.128291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:03:44.128291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:03:44.128291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:03:44.128291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:03:44.128291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 544/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:03:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:03:44.128895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d491e-70eb-4cd7-b39a-e19974c5fe81 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:44.134163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:44.233911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d491e-70eb-4cd7-b39a-e19974c5fe81 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:44.267174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028925c6-fecf-4f4c-8403-62dca25eb4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:44.267830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028925c6-fecf-4f4c-8403-62dca25eb4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:44.268112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028925c6-fecf-4f4c-8403-62dca25eb4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:44.268384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028925c6-fecf-4f4c-8403-62dca25eb4b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:44.269160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028925c6-fecf-4f4c-8403-62dca25eb4b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:44.269474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028925c6-fecf-4f4c-8403-62dca25eb4b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:44.270944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028925c6-fecf-4f4c-8403-62dca25eb4b3 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:44.282675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 561/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:03:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:44.288642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249baa8-ad0b-4f56-8df5-e331c4b95659 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:44.388774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249baa8-ad0b-4f56-8df5-e331c4b95659 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:44.423038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c69144-0e99-44dd-8645-5291a57201d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:44.423683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c69144-0e99-44dd-8645-5291a57201d7 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:44.423967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c69144-0e99-44dd-8645-5291a57201d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:44.424244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c69144-0e99-44dd-8645-5291a57201d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:44.425066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c69144-0e99-44dd-8645-5291a57201d7 None admin] RBAC: Adding query filter params (role.id=d538a3ae543f4c1897c8c3d9c8fb8066, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=bb6776311f354b748d563c4f1d793972) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:44.425378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c69144-0e99-44dd-8645-5291a57201d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:44.426912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c69144-0e99-44dd-8645-5291a57201d7 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:44.433129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 545/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:44 2022] GET /identity/v3/role_assignments?user.id=bb6776311f354b748d563c4f1d793972&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=d538a3ae543f4c1897c8c3d9c8fb8066 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:03:45.692324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd49c9a-d7c5-49c0-82d9-16f6d1c27d3d None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:45.692884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd49c9a-d7c5-49c0-82d9-16f6d1c27d3d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:45.692884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd49c9a-d7c5-49c0-82d9-16f6d1c27d3d None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:45.694249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 562/1107] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:45.700463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c220a4-1f2c-491d-9ee4-125a86002c73 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:45.700869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c220a4-1f2c-491d-9ee4-125a86002c73 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:45.701019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c220a4-1f2c-491d-9ee4-125a86002c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:45.718152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f0c220a4-1f2c-491d-9ee4-125a86002c73 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:45.719376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f0c220a4-1f2c-491d-9ee4-125a86002c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:45.760627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:45.783794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 546/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:45.809763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd49c9a-d7c5-49c0-82d9-16f6d1c27d3d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:45.871685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:45.909897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd49c9a-d7c5-49c0-82d9-16f6d1c27d3d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:45.943096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e7c41-c4b0-4e31-9a89-e6353ca80231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:45.943829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e7c41-c4b0-4e31-9a89-e6353ca80231 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:45.944112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e7c41-c4b0-4e31-9a89-e6353ca80231 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:45.944387 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e7c41-c4b0-4e31-9a89-e6353ca80231 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:45.946052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e7c41-c4b0-4e31-9a89-e6353ca80231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:45.947517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e7c41-c4b0-4e31-9a89-e6353ca80231 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:45.949136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 563/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:45.954973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c220a4-1f2c-491d-9ee4-125a86002c73 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:46.053354 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c220a4-1f2c-491d-9ee4-125a86002c73 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:46.087098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c24e4-832d-4218-b808-c36776d2f16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:46.087758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c24e4-832d-4218-b808-c36776d2f16c None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:46.088039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c24e4-832d-4218-b808-c36776d2f16c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:46.088310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c24e4-832d-4218-b808-c36776d2f16c None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:46.089049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c24e4-832d-4218-b808-c36776d2f16c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:46.090679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c24e4-832d-4218-b808-c36776d2f16c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:46.213167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 547/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:03:45 2022] POST /identity/v3/users => generated 286 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:03:48.494001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dbe53-1269-4c69-b893-a4c078e7d142 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:48.494557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dbe53-1269-4c69-b893-a4c078e7d142 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:48.494663 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dbe53-1269-4c69-b893-a4c078e7d142 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:48.495938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 564/1111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:48.501335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d7529-3a24-4f37-b589-a4ffe886bd3c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:48.501779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d7529-3a24-4f37-b589-a4ffe886bd3c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:48.501897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459d7529-3a24-4f37-b589-a4ffe886bd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:48.519212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-459d7529-3a24-4f37-b589-a4ffe886bd3c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:48.520392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-459d7529-3a24-4f37-b589-a4ffe886bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:48.546104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:48.587774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 548/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:48.611032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dbe53-1269-4c69-b893-a4c078e7d142 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:48.742665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dbe53-1269-4c69-b893-a4c078e7d142 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:48.771199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:48.775933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fa300-37fd-4dac-a193-c3a7260df7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:48.776794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fa300-37fd-4dac-a193-c3a7260df7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:48.777075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fa300-37fd-4dac-a193-c3a7260df7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:48.777352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fa300-37fd-4dac-a193-c3a7260df7b4 None admin] PATH_INFO: `/v3/roles/13607cfcced34c8badf88b8c244d32c5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:48.778840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fa300-37fd-4dac-a193-c3a7260df7b4 None admin] RBAC: Authorizing `identity:get_role(role_id=13607cfcced34c8badf88b8c244d32c5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:48.780217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fa300-37fd-4dac-a193-c3a7260df7b4 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:48.781578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 565/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:03:48 2022] GET /identity/v3/roles/13607cfcced34c8badf88b8c244d32c5 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:48.787443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d7529-3a24-4f37-b589-a4ffe886bd3c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:48.885220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459d7529-3a24-4f37-b589-a4ffe886bd3c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:48.918672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57184cae-0727-4927-8470-eac6619e0b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:48.919546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57184cae-0727-4927-8470-eac6619e0b43 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:48.919834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57184cae-0727-4927-8470-eac6619e0b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:48.920113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57184cae-0727-4927-8470-eac6619e0b43 None admin] PATH_INFO: `/v3/users/3829570a61dc4ad7b9451471a5085321` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:48.933872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57184cae-0727-4927-8470-eac6619e0b43 None admin] RBAC: Authorizing `identity:get_user(user_id=3829570a61dc4ad7b9451471a5085321)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:48.935343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57184cae-0727-4927-8470-eac6619e0b43 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:48.936986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 549/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:03:48 2022] GET /identity/v3/users/3829570a61dc4ad7b9451471a5085321 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:48.943037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fa300-37fd-4dac-a193-c3a7260df7b4 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:49.040940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fa300-37fd-4dac-a193-c3a7260df7b4 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:49.075840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d92f9-bbd1-4883-80f3-a6cb2219f865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:49.076480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d92f9-bbd1-4883-80f3-a6cb2219f865 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:49.076766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d92f9-bbd1-4883-80f3-a6cb2219f865 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:49.077038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d92f9-bbd1-4883-80f3-a6cb2219f865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:49.140409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:49.142621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d92f9-bbd1-4883-80f3-a6cb2219f865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:49.144087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d92f9-bbd1-4883-80f3-a6cb2219f865 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:49.239410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 566/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:03:49.246007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57184cae-0727-4927-8470-eac6619e0b43 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:49.267573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:49.342490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57184cae-0727-4927-8470-eac6619e0b43 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:49.375384 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8922a97-d0f4-4d7b-83ce-fe3e206df6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:49.376228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8922a97-d0f4-4d7b-83ce-fe3e206df6cb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:49.376509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8922a97-d0f4-4d7b-83ce-fe3e206df6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:49.376782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8922a97-d0f4-4d7b-83ce-fe3e206df6cb None admin] PATH_INFO: `/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:49.377936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8922a97-d0f4-4d7b-83ce-fe3e206df6cb None admin] RBAC: Authorizing `identity:get_project(project_id=3d2f600e49bc47e2a65ffb8fcda9d1fc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:49.379383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8922a97-d0f4-4d7b-83ce-fe3e206df6cb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:49.381039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 550/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:03:49 2022] GET /identity/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:49.388143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d92f9-bbd1-4883-80f3-a6cb2219f865 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:49.485755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:49.488548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d92f9-bbd1-4883-80f3-a6cb2219f865 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:49.523001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ff3ac-6fbd-47ca-8c9e-9c07cbab6369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:49.524136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ff3ac-6fbd-47ca-8c9e-9c07cbab6369 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:49.524422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ff3ac-6fbd-47ca-8c9e-9c07cbab6369 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:49.524697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ff3ac-6fbd-47ca-8c9e-9c07cbab6369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:49.525519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ff3ac-6fbd-47ca-8c9e-9c07cbab6369 None admin] RBAC: Adding query filter params (role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=3d2f600e49bc47e2a65ffb8fcda9d1fc, user.id=3829570a61dc4ad7b9451471a5085321) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:49.525830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ff3ac-6fbd-47ca-8c9e-9c07cbab6369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:49.527285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ff3ac-6fbd-47ca-8c9e-9c07cbab6369 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:49.532591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 567/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:49 2022] GET /identity/v3/role_assignments?user.id=3829570a61dc4ad7b9451471a5085321&scope.project.id=3d2f600e49bc47e2a65ffb8fcda9d1fc&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:51.733734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee642527-47ed-44cd-9fb3-cc241780cc9e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:51.734329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee642527-47ed-44cd-9fb3-cc241780cc9e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:51.734329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee642527-47ed-44cd-9fb3-cc241780cc9e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:51.735667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 551/1118] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:51.741047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407380bb-a20d-4d15-9ff0-ccdb0c7cf34f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:51.741478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407380bb-a20d-4d15-9ff0-ccdb0c7cf34f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:51.741614 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407380bb-a20d-4d15-9ff0-ccdb0c7cf34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:51.758695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-407380bb-a20d-4d15-9ff0-ccdb0c7cf34f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:51.759877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-407380bb-a20d-4d15-9ff0-ccdb0c7cf34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:51.823746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 568/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:51.847480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee642527-47ed-44cd-9fb3-cc241780cc9e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:51.934892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:51.947635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee642527-47ed-44cd-9fb3-cc241780cc9e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:51.980675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6a638-9f12-4cf1-a8b7-0490c953684e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:51.981515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6a638-9f12-4cf1-a8b7-0490c953684e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:51.981803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6a638-9f12-4cf1-a8b7-0490c953684e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:51.982074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6a638-9f12-4cf1-a8b7-0490c953684e None admin] PATH_INFO: `/v3/roles/13607cfcced34c8badf88b8c244d32c5` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:51.983565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6a638-9f12-4cf1-a8b7-0490c953684e None admin] RBAC: Authorizing `identity:get_role(role_id=13607cfcced34c8badf88b8c244d32c5)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:51.984976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6a638-9f12-4cf1-a8b7-0490c953684e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:51.986282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 552/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:03:51 2022] GET /identity/v3/roles/13607cfcced34c8badf88b8c244d32c5 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:51.992165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407380bb-a20d-4d15-9ff0-ccdb0c7cf34f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:52.084315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:52.091280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407380bb-a20d-4d15-9ff0-ccdb0c7cf34f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:52.124875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc25d8-d8fb-4bec-9f8a-db2bab918138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:52.125745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc25d8-d8fb-4bec-9f8a-db2bab918138 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:52.126026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc25d8-d8fb-4bec-9f8a-db2bab918138 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:52.126299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc25d8-d8fb-4bec-9f8a-db2bab918138 None admin] PATH_INFO: `/v3/users/3829570a61dc4ad7b9451471a5085321` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:52.127499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc25d8-d8fb-4bec-9f8a-db2bab918138 None admin] RBAC: Authorizing `identity:get_user(user_id=3829570a61dc4ad7b9451471a5085321)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:52.128923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc25d8-d8fb-4bec-9f8a-db2bab918138 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:52.130555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 569/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:03:51 2022] GET /identity/v3/users/3829570a61dc4ad7b9451471a5085321 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:52.136412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6a638-9f12-4cf1-a8b7-0490c953684e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:52.235257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6a638-9f12-4cf1-a8b7-0490c953684e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:52.268019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1049038-1409-40f4-b751-3570fa0351d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:52.268873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1049038-1409-40f4-b751-3570fa0351d5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:52.269187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1049038-1409-40f4-b751-3570fa0351d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:52.269455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1049038-1409-40f4-b751-3570fa0351d5 None admin] PATH_INFO: `/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:52.270645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1049038-1409-40f4-b751-3570fa0351d5 None admin] RBAC: Authorizing `identity:get_project(project_id=3d2f600e49bc47e2a65ffb8fcda9d1fc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:52.272084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1049038-1409-40f4-b751-3570fa0351d5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:52.274301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 553/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:03:52 2022] GET /identity/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:52.280173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc25d8-d8fb-4bec-9f8a-db2bab918138 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:52.376896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc25d8-d8fb-4bec-9f8a-db2bab918138 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:52.410172 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1588f3-35cc-4f2d-a72c-cebd0727ceb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:52.411068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1588f3-35cc-4f2d-a72c-cebd0727ceb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:52.411350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1588f3-35cc-4f2d-a72c-cebd0727ceb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:52.411626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1588f3-35cc-4f2d-a72c-cebd0727ceb7 None admin] PATH_INFO: `/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc/users/3829570a61dc4ad7b9451471a5085321/roles/13607cfcced34c8badf88b8c244d32c5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:52.414489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1588f3-35cc-4f2d-a72c-cebd0727ceb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d2f600e49bc47e2a65ffb8fcda9d1fc, user_id=3829570a61dc4ad7b9451471a5085321, role_id=13607cfcced34c8badf88b8c244d32c5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:52.416253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1588f3-35cc-4f2d-a72c-cebd0727ceb7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:52.510541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 570/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:03:52 2022] PUT /identity/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc/users/3829570a61dc4ad7b9451471a5085321/roles/13607cfcced34c8badf88b8c244d32c5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:03:54.721224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc4649-d05a-4c7b-a291-1ad57201dad7 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:54.721777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc4649-d05a-4c7b-a291-1ad57201dad7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:54.721777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc4649-d05a-4c7b-a291-1ad57201dad7 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:54.723152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 554/1124] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:54.728891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c7aa1-fcb7-48f7-bded-64bb2a86ca6e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:54.729304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c7aa1-fcb7-48f7-bded-64bb2a86ca6e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:54.729443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c7aa1-fcb7-48f7-bded-64bb2a86ca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:54.746196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e66c7aa1-fcb7-48f7-bded-64bb2a86ca6e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:54.747399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e66c7aa1-fcb7-48f7-bded-64bb2a86ca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:54.763122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:54.813729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 571/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:54.837144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc4649-d05a-4c7b-a291-1ad57201dad7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:54.867510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:54.969832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc4649-d05a-4c7b-a291-1ad57201dad7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:55.004759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc704bf-5a63-4985-8bb3-4d308707fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:55.005554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc704bf-5a63-4985-8bb3-4d308707fed8 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:55.005838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc704bf-5a63-4985-8bb3-4d308707fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:55.006116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc704bf-5a63-4985-8bb3-4d308707fed8 None admin] PATH_INFO: `/v3/roles/13607cfcced34c8badf88b8c244d32c5` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:55.007690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc704bf-5a63-4985-8bb3-4d308707fed8 None admin] RBAC: Authorizing `identity:get_role(role_id=13607cfcced34c8badf88b8c244d32c5)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:55.009052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc704bf-5a63-4985-8bb3-4d308707fed8 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:55.010442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 555/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:03:54 2022] GET /identity/v3/roles/13607cfcced34c8badf88b8c244d32c5 => generated 223 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:55.016372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c7aa1-fcb7-48f7-bded-64bb2a86ca6e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:55.114973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c7aa1-fcb7-48f7-bded-64bb2a86ca6e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:55.147541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cb925-2636-4901-878b-8a4369b1d132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:55.149043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cb925-2636-4901-878b-8a4369b1d132 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:55.149326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cb925-2636-4901-878b-8a4369b1d132 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:55.149608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cb925-2636-4901-878b-8a4369b1d132 None admin] PATH_INFO: `/v3/users/3829570a61dc4ad7b9451471a5085321` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:55.150834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cb925-2636-4901-878b-8a4369b1d132 None admin] RBAC: Authorizing `identity:get_user(user_id=3829570a61dc4ad7b9451471a5085321)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:55.152273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cb925-2636-4901-878b-8a4369b1d132 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:55.153955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 572/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:03:55 2022] GET /identity/v3/users/3829570a61dc4ad7b9451471a5085321 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:55.159789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc704bf-5a63-4985-8bb3-4d308707fed8 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:55.259709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:55.268693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc704bf-5a63-4985-8bb3-4d308707fed8 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:55.302811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57482360-a7ea-44dc-971a-4f539ea4778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:55.303455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482360-a7ea-44dc-971a-4f539ea4778b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:55.303736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482360-a7ea-44dc-971a-4f539ea4778b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:55.304494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482360-a7ea-44dc-971a-4f539ea4778b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:55.369244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57482360-a7ea-44dc-971a-4f539ea4778b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:55.370714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57482360-a7ea-44dc-971a-4f539ea4778b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:55.458135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:55.467383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 556/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:03:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:03:55.473341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cb925-2636-4901-878b-8a4369b1d132 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:55.496421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:55.575366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cb925-2636-4901-878b-8a4369b1d132 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:55.608537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b18919-adf2-49f2-91e3-d75de9e24ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:55.610805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18919-adf2-49f2-91e3-d75de9e24ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:55.610805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18919-adf2-49f2-91e3-d75de9e24ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:55.610805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b18919-adf2-49f2-91e3-d75de9e24ad7 None admin] PATH_INFO: `/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:55.611155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b18919-adf2-49f2-91e3-d75de9e24ad7 None admin] RBAC: Authorizing `identity:get_project(project_id=3d2f600e49bc47e2a65ffb8fcda9d1fc)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:55.612534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b18919-adf2-49f2-91e3-d75de9e24ad7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:55.614214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 573/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:03:55 2022] GET /identity/v3/projects/3d2f600e49bc47e2a65ffb8fcda9d1fc => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:55.620447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57482360-a7ea-44dc-971a-4f539ea4778b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:55.719138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57482360-a7ea-44dc-971a-4f539ea4778b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:55.753006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702896fb-f373-45f7-bc5c-1fadddeaf361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:55.753665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702896fb-f373-45f7-bc5c-1fadddeaf361 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:55.753919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702896fb-f373-45f7-bc5c-1fadddeaf361 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:55.754196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702896fb-f373-45f7-bc5c-1fadddeaf361 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:55.755052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702896fb-f373-45f7-bc5c-1fadddeaf361 None admin] RBAC: Adding query filter params (role.id=13607cfcced34c8badf88b8c244d32c5, scope.project.id=3d2f600e49bc47e2a65ffb8fcda9d1fc, user.id=3829570a61dc4ad7b9451471a5085321) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:03:55.755369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702896fb-f373-45f7-bc5c-1fadddeaf361 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:55.756810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702896fb-f373-45f7-bc5c-1fadddeaf361 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:55.763067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 557/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:03:55 2022] GET /identity/v3/role_assignments?user.id=3829570a61dc4ad7b9451471a5085321&scope.project.id=3d2f600e49bc47e2a65ffb8fcda9d1fc&role.id=13607cfcced34c8badf88b8c244d32c5 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:58.018616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ce722-072f-45db-9169-49ece43afb8a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:58.019171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ce722-072f-45db-9169-49ece43afb8a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:58.019171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ce722-072f-45db-9169-49ece43afb8a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:58.020581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 574/1131] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:03:58.026376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bddfb-0d42-413c-9484-38810932dbf2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:58.026822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bddfb-0d42-413c-9484-38810932dbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:58.026985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bddfb-0d42-413c-9484-38810932dbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:58.044266 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b87bddfb-0d42-413c-9484-38810932dbf2 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:03:58.045435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b87bddfb-0d42-413c-9484-38810932dbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:03:58.095026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:58.111606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 558/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:03:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:03:58.134620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ce722-072f-45db-9169-49ece43afb8a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:58.220236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:03:58.232775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ce722-072f-45db-9169-49ece43afb8a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:58.265340 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d34fe-c0d3-47a4-8a13-956c9fe00b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:58.266117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d34fe-c0d3-47a4-8a13-956c9fe00b21 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:58.266397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d34fe-c0d3-47a4-8a13-956c9fe00b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:58.266694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d34fe-c0d3-47a4-8a13-956c9fe00b21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:58.267860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d34fe-c0d3-47a4-8a13-956c9fe00b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:58.269326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d34fe-c0d3-47a4-8a13-956c9fe00b21 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:58.270922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 575/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:03:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:03:58.277239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bddfb-0d42-413c-9484-38810932dbf2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:03:58.375158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bddfb-0d42-413c-9484-38810932dbf2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:03:58.408745 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f6a80-2f53-4069-a913-01ca5e47635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:03:58.409399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f6a80-2f53-4069-a913-01ca5e47635e None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:03:58.409686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f6a80-2f53-4069-a913-01ca5e47635e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:03:58.409963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f6a80-2f53-4069-a913-01ca5e47635e None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:03:58.410778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f6a80-2f53-4069-a913-01ca5e47635e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:03:58.412217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f6a80-2f53-4069-a913-01ca5e47635e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:03:58.554602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 559/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:03:58 2022] POST /identity/v3/projects => generated 311 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:04:00.887708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736c074-2873-4897-810e-1cf1b0017b74 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:00.888264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736c074-2873-4897-810e-1cf1b0017b74 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:00.888264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736c074-2873-4897-810e-1cf1b0017b74 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:00.889603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 576/1135] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:00.894768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e7ef6-1393-4757-a551-5fcb6f210cad None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:00.895167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e7ef6-1393-4757-a551-5fcb6f210cad None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:00.895338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e7ef6-1393-4757-a551-5fcb6f210cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:00.912485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-bb7e7ef6-1393-4757-a551-5fcb6f210cad None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:00.913667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-bb7e7ef6-1393-4757-a551-5fcb6f210cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:00.944273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:00.980971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 560/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:01.005721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736c074-2873-4897-810e-1cf1b0017b74 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:01.108308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736c074-2873-4897-810e-1cf1b0017b74 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:01.145145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd3355-1290-4244-9ed3-eb37c75ab7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:01.145850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd3355-1290-4244-9ed3-eb37c75ab7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:01.146140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd3355-1290-4244-9ed3-eb37c75ab7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:01.146439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd3355-1290-4244-9ed3-eb37c75ab7d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:01.147640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd3355-1290-4244-9ed3-eb37c75ab7d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:01.149308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd3355-1290-4244-9ed3-eb37c75ab7d8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:01.151300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 577/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:04:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:01.156934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e7ef6-1393-4757-a551-5fcb6f210cad None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:01.253319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e7ef6-1393-4757-a551-5fcb6f210cad None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:01.287881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3354ede-dc26-4ab5-91f6-ee669753382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:01.288547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3354ede-dc26-4ab5-91f6-ee669753382e None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:01.288829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3354ede-dc26-4ab5-91f6-ee669753382e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:01.289104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3354ede-dc26-4ab5-91f6-ee669753382e None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:01.290557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3354ede-dc26-4ab5-91f6-ee669753382e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:01.292240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3354ede-dc26-4ab5-91f6-ee669753382e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:01.421787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 561/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:04:01 2022] POST /identity/v3/users => generated 286 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:04:03.741922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b0a56-bb6a-4270-887c-5c04137895dc None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:03.742517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b0a56-bb6a-4270-887c-5c04137895dc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:03.743324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b0a56-bb6a-4270-887c-5c04137895dc None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:03.745672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 578/1139] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:03.751240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8c7ec-186e-4c73-b2f7-8084c35d7522 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:03.751653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8c7ec-186e-4c73-b2f7-8084c35d7522 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:03.751810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8c7ec-186e-4c73-b2f7-8084c35d7522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:03.768463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-86f8c7ec-186e-4c73-b2f7-8084c35d7522 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:03.769652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-86f8c7ec-186e-4c73-b2f7-8084c35d7522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:03.786783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:03.835933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 562/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:03.860653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b0a56-bb6a-4270-887c-5c04137895dc None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:03.886606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:03.960517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b0a56-bb6a-4270-887c-5c04137895dc None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:03.995126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3143c-ef54-4afa-94c2-eb925c953963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:03.995843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3143c-ef54-4afa-94c2-eb925c953963 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:03.996125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3143c-ef54-4afa-94c2-eb925c953963 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:03.996397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3143c-ef54-4afa-94c2-eb925c953963 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:04.006255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3143c-ef54-4afa-94c2-eb925c953963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-6ab3143c-ef54-4afa-94c2-eb925c953963 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:04:04.014307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:04:04.015514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:04:04.016772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 579/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:04:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:04:04.021523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8c7ec-186e-4c73-b2f7-8084c35d7522 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:04.121117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8c7ec-186e-4c73-b2f7-8084c35d7522 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:04.155523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48982ef6-c823-4b77-8136-de45ec64f61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:04.156173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48982ef6-c823-4b77-8136-de45ec64f61c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:04.156460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48982ef6-c823-4b77-8136-de45ec64f61c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:04.156735 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48982ef6-c823-4b77-8136-de45ec64f61c None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:04.157478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48982ef6-c823-4b77-8136-de45ec64f61c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:04.157789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48982ef6-c823-4b77-8136-de45ec64f61c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:04.159201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48982ef6-c823-4b77-8136-de45ec64f61c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:04.168044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 563/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:04:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:04.173805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3143c-ef54-4afa-94c2-eb925c953963 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:04.254913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:04.269731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3143c-ef54-4afa-94c2-eb925c953963 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:04.303890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a29e5-4403-4f53-b95b-580f58040558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:04.304764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a29e5-4403-4f53-b95b-580f58040558 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:04.305047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a29e5-4403-4f53-b95b-580f58040558 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:04.305449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a29e5-4403-4f53-b95b-580f58040558 None admin] PATH_INFO: `/v3/users/b8052a8eb8504e11b9d63c232f9bc342` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:04.318248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a29e5-4403-4f53-b95b-580f58040558 None admin] RBAC: Authorizing `identity:get_user(user_id=b8052a8eb8504e11b9d63c232f9bc342)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:04.319704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a29e5-4403-4f53-b95b-580f58040558 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:04.321674 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 580/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:04:04 2022] GET /identity/v3/users/b8052a8eb8504e11b9d63c232f9bc342 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:04.327382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48982ef6-c823-4b77-8136-de45ec64f61c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:04.346191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:04.427563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48982ef6-c823-4b77-8136-de45ec64f61c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:04.459863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd031518-2963-42bd-bd65-f23b73a486ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:04.460506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd031518-2963-42bd-bd65-f23b73a486ca None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:04.460790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd031518-2963-42bd-bd65-f23b73a486ca None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:04.461130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd031518-2963-42bd-bd65-f23b73a486ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:04.526437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd031518-2963-42bd-bd65-f23b73a486ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:04.527808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd031518-2963-42bd-bd65-f23b73a486ca None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:04.544766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:04.625698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 564/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:04:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:04:04.631582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a29e5-4403-4f53-b95b-580f58040558 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:04.729293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a29e5-4403-4f53-b95b-580f58040558 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:04.762312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30004e3-3540-4028-9ab7-c576290474e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:04.763196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30004e3-3540-4028-9ab7-c576290474e8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:04.763477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30004e3-3540-4028-9ab7-c576290474e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:04.763751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30004e3-3540-4028-9ab7-c576290474e8 None admin] PATH_INFO: `/v3/projects/fcf344089abb4688b62bb7f06191f609` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:04.764921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30004e3-3540-4028-9ab7-c576290474e8 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf344089abb4688b62bb7f06191f609)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:04.766344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30004e3-3540-4028-9ab7-c576290474e8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:04.768046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 581/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:04:04 2022] GET /identity/v3/projects/fcf344089abb4688b62bb7f06191f609 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:04.774924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd031518-2963-42bd-bd65-f23b73a486ca None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:04.872529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd031518-2963-42bd-bd65-f23b73a486ca None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:04.889011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:04.906373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c973413-303e-4463-b29f-d8e397db06f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:04.907062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c973413-303e-4463-b29f-d8e397db06f6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:04.907347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c973413-303e-4463-b29f-d8e397db06f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:04.907625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c973413-303e-4463-b29f-d8e397db06f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:04.908455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c973413-303e-4463-b29f-d8e397db06f6 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=fcf344089abb4688b62bb7f06191f609, user.id=b8052a8eb8504e11b9d63c232f9bc342) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:04.908771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c973413-303e-4463-b29f-d8e397db06f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:04.910203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c973413-303e-4463-b29f-d8e397db06f6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:04.916089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 565/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:04:04 2022] GET /identity/v3/role_assignments?user.id=b8052a8eb8504e11b9d63c232f9bc342&scope.project.id=fcf344089abb4688b62bb7f06191f609&role.id=bd295cb5d1194b469beff0381c6b23af => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:07.151814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374206d3-c649-4f2e-ae90-544f3593790f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:07.152338 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374206d3-c649-4f2e-ae90-544f3593790f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:07.152447 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374206d3-c649-4f2e-ae90-544f3593790f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:07.153716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 582/1147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:07.158996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40477b-0ec2-47bf-8965-f36151b6e7ae None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:07.159401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40477b-0ec2-47bf-8965-f36151b6e7ae None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:07.159555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40477b-0ec2-47bf-8965-f36151b6e7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:07.176764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2a40477b-0ec2-47bf-8965-f36151b6e7ae None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:07.177933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2a40477b-0ec2-47bf-8965-f36151b6e7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:07.244727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 566/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 12:04:07.267174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374206d3-c649-4f2e-ae90-544f3593790f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:07.280057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:07.370100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374206d3-c649-4f2e-ae90-544f3593790f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:07.403765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd390c-4c57-465b-ae30-4d6a1116bd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:07.404593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd390c-4c57-465b-ae30-4d6a1116bd46 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:07.404878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd390c-4c57-465b-ae30-4d6a1116bd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:07.405154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd390c-4c57-465b-ae30-4d6a1116bd46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:07.415696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd390c-4c57-465b-ae30-4d6a1116bd46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-5edd390c-4c57-465b-ae30-4d6a1116bd46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:04:07.423618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:04:07.424699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:04:07.425580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 583/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:04:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:04:07.430982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40477b-0ec2-47bf-8965-f36151b6e7ae None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:07.529367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40477b-0ec2-47bf-8965-f36151b6e7ae None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:07.540159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:07.565527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28349598-c602-44df-aa2e-8113b5b26ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:07.566188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28349598-c602-44df-aa2e-8113b5b26ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:07.566498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28349598-c602-44df-aa2e-8113b5b26ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:07.566780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28349598-c602-44df-aa2e-8113b5b26ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:07.567581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28349598-c602-44df-aa2e-8113b5b26ea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:07.567890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28349598-c602-44df-aa2e-8113b5b26ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:07.569254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28349598-c602-44df-aa2e-8113b5b26ea9 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:07.577759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 567/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:04:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:07.583708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd390c-4c57-465b-ae30-4d6a1116bd46 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:07.654115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:07.684258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd390c-4c57-465b-ae30-4d6a1116bd46 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:07.718052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7df4df-4857-4b86-afae-2ff35db34a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:07.718943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7df4df-4857-4b86-afae-2ff35db34a21 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:07.719223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7df4df-4857-4b86-afae-2ff35db34a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:07.719517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7df4df-4857-4b86-afae-2ff35db34a21 None admin] PATH_INFO: `/v3/users/b8052a8eb8504e11b9d63c232f9bc342` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:07.720679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7df4df-4857-4b86-afae-2ff35db34a21 None admin] RBAC: Authorizing `identity:get_user(user_id=b8052a8eb8504e11b9d63c232f9bc342)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:07.722090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7df4df-4857-4b86-afae-2ff35db34a21 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:07.723748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 584/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:04:07 2022] GET /identity/v3/users/b8052a8eb8504e11b9d63c232f9bc342 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:07.730025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28349598-c602-44df-aa2e-8113b5b26ea9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:07.829291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28349598-c602-44df-aa2e-8113b5b26ea9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:07.862836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05322e27-f346-42ab-8e02-87d2fb268378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:07.863686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05322e27-f346-42ab-8e02-87d2fb268378 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:07.863966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05322e27-f346-42ab-8e02-87d2fb268378 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:07.864240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05322e27-f346-42ab-8e02-87d2fb268378 None admin] PATH_INFO: `/v3/projects/fcf344089abb4688b62bb7f06191f609` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:07.865447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05322e27-f346-42ab-8e02-87d2fb268378 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf344089abb4688b62bb7f06191f609)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:07.866903 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05322e27-f346-42ab-8e02-87d2fb268378 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:07.869078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 568/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:04:07 2022] GET /identity/v3/projects/fcf344089abb4688b62bb7f06191f609 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:07.875012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7df4df-4857-4b86-afae-2ff35db34a21 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:07.971667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7df4df-4857-4b86-afae-2ff35db34a21 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:08.003758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2878d3-18fc-4087-847f-2fd0ae4d2932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:08.004706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2878d3-18fc-4087-847f-2fd0ae4d2932 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:08.004989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2878d3-18fc-4087-847f-2fd0ae4d2932 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:08.005262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2878d3-18fc-4087-847f-2fd0ae4d2932 None admin] PATH_INFO: `/v3/projects/fcf344089abb4688b62bb7f06191f609/users/b8052a8eb8504e11b9d63c232f9bc342/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:08.007201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2878d3-18fc-4087-847f-2fd0ae4d2932 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf344089abb4688b62bb7f06191f609, user_id=b8052a8eb8504e11b9d63c232f9bc342, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:08.009319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2878d3-18fc-4087-847f-2fd0ae4d2932 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:08.092493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:08.097883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 585/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:04:07 2022] PUT /identity/v3/projects/fcf344089abb4688b62bb7f06191f609/users/b8052a8eb8504e11b9d63c232f9bc342/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:04:10.287976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a83992-6b2f-4b74-a73a-acc2a4bfe951 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:10.288479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a83992-6b2f-4b74-a73a-acc2a4bfe951 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:10.288479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a83992-6b2f-4b74-a73a-acc2a4bfe951 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:10.289863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 569/1154] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:10.295419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b889b6a-4531-4c9a-9bbf-ee7d26590ff9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:10.295794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b889b6a-4531-4c9a-9bbf-ee7d26590ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:10.295983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b889b6a-4531-4c9a-9bbf-ee7d26590ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:10.312854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4b889b6a-4531-4c9a-9bbf-ee7d26590ff9 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:10.314032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4b889b6a-4531-4c9a-9bbf-ee7d26590ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:10.379503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 586/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:10.403945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a83992-6b2f-4b74-a73a-acc2a4bfe951 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:10.447308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:10.512730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a83992-6b2f-4b74-a73a-acc2a4bfe951 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:10.554546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fde4a-d5c2-415d-b8aa-c7bdab16e483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:10.556092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fde4a-d5c2-415d-b8aa-c7bdab16e483 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:10.556376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fde4a-d5c2-415d-b8aa-c7bdab16e483 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:10.556650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fde4a-d5c2-415d-b8aa-c7bdab16e483 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:10.567057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fde4a-d5c2-415d-b8aa-c7bdab16e483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-2d3fde4a-d5c2-415d-b8aa-c7bdab16e483 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:04:10.575717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:04:10.576650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:04:10.577625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 570/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:04:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:04:10.582970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b889b6a-4531-4c9a-9bbf-ee7d26590ff9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:10.681114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b889b6a-4531-4c9a-9bbf-ee7d26590ff9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:10.706990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:10.715605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8b8e2-14c0-40ec-a865-b674eca9d67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:10.718545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8b8e2-14c0-40ec-a865-b674eca9d67c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:10.718545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8b8e2-14c0-40ec-a865-b674eca9d67c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:10.718545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8b8e2-14c0-40ec-a865-b674eca9d67c None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:10.718545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8b8e2-14c0-40ec-a865-b674eca9d67c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:10.718545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8b8e2-14c0-40ec-a865-b674eca9d67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:10.719217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8b8e2-14c0-40ec-a865-b674eca9d67c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:10.727199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 587/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:04:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:10.733119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fde4a-d5c2-415d-b8aa-c7bdab16e483 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:10.831222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:10.834300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fde4a-d5c2-415d-b8aa-c7bdab16e483 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:10.871856 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc1f5-7187-4174-88f9-787a1b217b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:10.872739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc1f5-7187-4174-88f9-787a1b217b76 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:10.873300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc1f5-7187-4174-88f9-787a1b217b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:10.873582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc1f5-7187-4174-88f9-787a1b217b76 None admin] PATH_INFO: `/v3/users/b8052a8eb8504e11b9d63c232f9bc342` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:10.874796 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edc1f5-7187-4174-88f9-787a1b217b76 None admin] RBAC: Authorizing `identity:get_user(user_id=b8052a8eb8504e11b9d63c232f9bc342)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:10.876230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7edc1f5-7187-4174-88f9-787a1b217b76 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:10.877852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 571/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:04:10 2022] GET /identity/v3/users/b8052a8eb8504e11b9d63c232f9bc342 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:10.884305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8b8e2-14c0-40ec-a865-b674eca9d67c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:10.982075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8b8e2-14c0-40ec-a865-b674eca9d67c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:11.015246 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15180743-cdac-45ba-beea-b766672c44b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:11.015949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15180743-cdac-45ba-beea-b766672c44b8 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:11.016214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15180743-cdac-45ba-beea-b766672c44b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:11.016485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15180743-cdac-45ba-beea-b766672c44b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:11.074874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:11.101966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15180743-cdac-45ba-beea-b766672c44b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:11.101966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15180743-cdac-45ba-beea-b766672c44b8 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:11.185570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 588/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:04:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:04:11.191593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc1f5-7187-4174-88f9-787a1b217b76 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:11.289939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc1f5-7187-4174-88f9-787a1b217b76 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:11.325347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697aebc-9d56-432a-bc07-d609c4c3819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:11.326189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697aebc-9d56-432a-bc07-d609c4c3819a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:11.326495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697aebc-9d56-432a-bc07-d609c4c3819a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:11.326768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697aebc-9d56-432a-bc07-d609c4c3819a None admin] PATH_INFO: `/v3/projects/fcf344089abb4688b62bb7f06191f609` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:11.327931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697aebc-9d56-432a-bc07-d609c4c3819a None admin] RBAC: Authorizing `identity:get_project(project_id=fcf344089abb4688b62bb7f06191f609)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:11.329386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697aebc-9d56-432a-bc07-d609c4c3819a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:11.331076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 572/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:04:11 2022] GET /identity/v3/projects/fcf344089abb4688b62bb7f06191f609 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:11.338074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15180743-cdac-45ba-beea-b766672c44b8 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:11.426647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:11.436562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15180743-cdac-45ba-beea-b766672c44b8 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:11.469331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf5435-9477-474c-8132-afe2c838124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:11.469969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf5435-9477-474c-8132-afe2c838124f None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:11.470254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf5435-9477-474c-8132-afe2c838124f None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:11.470559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf5435-9477-474c-8132-afe2c838124f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:11.471386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf5435-9477-474c-8132-afe2c838124f None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=fcf344089abb4688b62bb7f06191f609, user.id=b8052a8eb8504e11b9d63c232f9bc342) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:11.471701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf5435-9477-474c-8132-afe2c838124f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:11.473128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf5435-9477-474c-8132-afe2c838124f None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:11.479420 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 589/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:04:11 2022] GET /identity/v3/role_assignments?user.id=b8052a8eb8504e11b9d63c232f9bc342&scope.project.id=fcf344089abb4688b62bb7f06191f609&role.id=bd295cb5d1194b469beff0381c6b23af => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:13.732029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12680826-dc98-47e1-b6ff-970133ebe5e4 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:13.732588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12680826-dc98-47e1-b6ff-970133ebe5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:13.732588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12680826-dc98-47e1-b6ff-970133ebe5e4 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:13.733922 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 573/1162] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:13.739340 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4a4c6-4e69-4582-82c3-823b1675aa68 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:13.739721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4a4c6-4e69-4582-82c3-823b1675aa68 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:13.739900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4a4c6-4e69-4582-82c3-823b1675aa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:13.757352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d1a4a4c6-4e69-4582-82c3-823b1675aa68 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:13.758599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d1a4a4c6-4e69-4582-82c3-823b1675aa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:13.825633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 590/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:13.848572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12680826-dc98-47e1-b6ff-970133ebe5e4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:13.886145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:13.951215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12680826-dc98-47e1-b6ff-970133ebe5e4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:13.985136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431bd8-797c-4489-be5b-cf4ea76ae28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:13.985705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431bd8-797c-4489-be5b-cf4ea76ae28c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:13.985987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431bd8-797c-4489-be5b-cf4ea76ae28c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:13.986260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431bd8-797c-4489-be5b-cf4ea76ae28c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:14.052271 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36431bd8-797c-4489-be5b-cf4ea76ae28c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:14.053736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36431bd8-797c-4489-be5b-cf4ea76ae28c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:14.087831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:14.151156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 574/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:04:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:04:14.157256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4a4c6-4e69-4582-82c3-823b1675aa68 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:14.239420 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:14.256040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4a4c6-4e69-4582-82c3-823b1675aa68 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:14.291065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33afc56-d89b-4ee1-a257-2607c31b8c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:14.291865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33afc56-d89b-4ee1-a257-2607c31b8c30 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:14.292149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33afc56-d89b-4ee1-a257-2607c31b8c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:14.292421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33afc56-d89b-4ee1-a257-2607c31b8c30 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:14.299054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33afc56-d89b-4ee1-a257-2607c31b8c30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:14.300527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33afc56-d89b-4ee1-a257-2607c31b8c30 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a33afc56-d89b-4ee1-a257-2607c31b8c30 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:04:14.314999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:04:14.316279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:04:14.317132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 591/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:04:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:04:14.322586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431bd8-797c-4489-be5b-cf4ea76ae28c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:14.419710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431bd8-797c-4489-be5b-cf4ea76ae28c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:14.452879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbd5b8-adb0-4050-9da4-023f40862cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:14.453536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd5b8-adb0-4050-9da4-023f40862cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:14.453820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd5b8-adb0-4050-9da4-023f40862cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:14.454096 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd5b8-adb0-4050-9da4-023f40862cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:14.454841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd5b8-adb0-4050-9da4-023f40862cb5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:14.455150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd5b8-adb0-4050-9da4-023f40862cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:14.456508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd5b8-adb0-4050-9da4-023f40862cb5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:14.458766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:14.463809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 575/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 19 12:04:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:16.672953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d9363-21b6-4c80-a217-1178ff7873d4 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:16.673510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d9363-21b6-4c80-a217-1178ff7873d4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:16.673510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d9363-21b6-4c80-a217-1178ff7873d4 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:16.674941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 592/1167] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:16.680196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012bf54d-da0f-4354-97d1-9e667abf579c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:16.680617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012bf54d-da0f-4354-97d1-9e667abf579c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:16.680837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012bf54d-da0f-4354-97d1-9e667abf579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:16.697869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-012bf54d-da0f-4354-97d1-9e667abf579c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:16.699067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-012bf54d-da0f-4354-97d1-9e667abf579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:16.764173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 576/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:16.787440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d9363-21b6-4c80-a217-1178ff7873d4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:16.890998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d9363-21b6-4c80-a217-1178ff7873d4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:16.925671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e05e67-9d38-4d85-9b2d-e7b7c9629427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:16.926274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e05e67-9d38-4d85-9b2d-e7b7c9629427 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:16.926582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e05e67-9d38-4d85-9b2d-e7b7c9629427 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:16.926860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e05e67-9d38-4d85-9b2d-e7b7c9629427 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:16.927603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e05e67-9d38-4d85-9b2d-e7b7c9629427 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:16.928998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e05e67-9d38-4d85-9b2d-e7b7c9629427 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:17.012645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 593/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu May 19 12:04:16 2022] POST /identity/v3/domains => generated 267 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:04:19.286048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934315-e96e-40ea-94f8-9dc827cd9e46 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:19.286604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934315-e96e-40ea-94f8-9dc827cd9e46 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:19.286604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934315-e96e-40ea-94f8-9dc827cd9e46 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:19.287894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 577/1170] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:19.294461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590a310-9d87-4b91-8706-2b880a41c3cc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:19.294843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590a310-9d87-4b91-8706-2b880a41c3cc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:19.295018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590a310-9d87-4b91-8706-2b880a41c3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:19.300915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:19.311903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d590a310-9d87-4b91-8706-2b880a41c3cc None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:19.317413 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d590a310-9d87-4b91-8706-2b880a41c3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:19.378191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 594/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:19.403125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934315-e96e-40ea-94f8-9dc827cd9e46 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:19.504633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7934315-e96e-40ea-94f8-9dc827cd9e46 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:19.532390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:19.540977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8a616-0601-4cdb-966c-3445b5c18268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:19.540977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8a616-0601-4cdb-966c-3445b5c18268 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:19.540977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8a616-0601-4cdb-966c-3445b5c18268 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:19.540977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8a616-0601-4cdb-966c-3445b5c18268 None admin] PATH_INFO: `/v3/domains/0ef7d195a5e44eaa8e2b5548f9b100ba` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:19.550623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8a616-0601-4cdb-966c-3445b5c18268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ef7d195a5e44eaa8e2b5548f9b100ba)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:19.552080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8a616-0601-4cdb-966c-3445b5c18268 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:19.554539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 578/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 12:04:19 2022] GET /identity/v3/domains/0ef7d195a5e44eaa8e2b5548f9b100ba => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:19.560368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590a310-9d87-4b91-8706-2b880a41c3cc None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:19.655220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590a310-9d87-4b91-8706-2b880a41c3cc None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:19.688960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73047035-d617-4c0e-b86d-e71d67541aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:19.689612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73047035-d617-4c0e-b86d-e71d67541aef None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:19.689902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73047035-d617-4c0e-b86d-e71d67541aef None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:19.690176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73047035-d617-4c0e-b86d-e71d67541aef None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:19.690908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73047035-d617-4c0e-b86d-e71d67541aef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:19.692334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73047035-d617-4c0e-b86d-e71d67541aef None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:19.695560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:19.829250 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 595/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:04:19 2022] POST /identity/v3/projects => generated 361 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:04:22.115220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49daea-b603-4737-957b-b6eea65f0e46 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:22.115795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49daea-b603-4737-957b-b6eea65f0e46 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:22.115795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49daea-b603-4737-957b-b6eea65f0e46 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:22.117842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 579/1174] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:22.123304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d2ed2-b7f8-4ed2-aa28-dbafd42d8e5a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:22.123727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d2ed2-b7f8-4ed2-aa28-dbafd42d8e5a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:22.123865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d2ed2-b7f8-4ed2-aa28-dbafd42d8e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:22.140887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-242d2ed2-b7f8-4ed2-aa28-dbafd42d8e5a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:22.142068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-242d2ed2-b7f8-4ed2-aa28-dbafd42d8e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:22.207528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 596/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:22.232822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49daea-b603-4737-957b-b6eea65f0e46 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:22.335104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49daea-b603-4737-957b-b6eea65f0e46 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:22.368056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc5bb0-2653-47c0-a0e7-4ade9c738d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:22.368907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc5bb0-2653-47c0-a0e7-4ade9c738d16 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:22.369189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc5bb0-2653-47c0-a0e7-4ade9c738d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:22.369462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc5bb0-2653-47c0-a0e7-4ade9c738d16 None admin] PATH_INFO: `/v3/domains/0ef7d195a5e44eaa8e2b5548f9b100ba` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:22.370641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc5bb0-2653-47c0-a0e7-4ade9c738d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ef7d195a5e44eaa8e2b5548f9b100ba)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:22.372055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc5bb0-2653-47c0-a0e7-4ade9c738d16 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:22.373761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 580/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 19 12:04:22 2022] GET /identity/v3/domains/0ef7d195a5e44eaa8e2b5548f9b100ba => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:22.379618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d2ed2-b7f8-4ed2-aa28-dbafd42d8e5a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:22.476603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d2ed2-b7f8-4ed2-aa28-dbafd42d8e5a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:22.493089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:22.509858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b138d-12a8-4d9a-8a95-ceddde079f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:22.510557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b138d-12a8-4d9a-8a95-ceddde079f29 None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:22.510846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b138d-12a8-4d9a-8a95-ceddde079f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:22.511122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b138d-12a8-4d9a-8a95-ceddde079f29 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:22.511836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b138d-12a8-4d9a-8a95-ceddde079f29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:22.513319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b138d-12a8-4d9a-8a95-ceddde079f29 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:22.666901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 597/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:04:22 2022] POST /identity/v3/users => generated 311 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:04:24.966037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc324b5-d6c9-491d-95cc-1e24cfc904f9 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:24.966654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc324b5-d6c9-491d-95cc-1e24cfc904f9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:24.966654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc324b5-d6c9-491d-95cc-1e24cfc904f9 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:24.968002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 581/1178] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:24.973493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af192bf-2289-4c1b-8ab8-b6805885ba7d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:24.973935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af192bf-2289-4c1b-8ab8-b6805885ba7d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:24.974045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af192bf-2289-4c1b-8ab8-b6805885ba7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:24.991782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4af192bf-2289-4c1b-8ab8-b6805885ba7d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:24.993136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4af192bf-2289-4c1b-8ab8-b6805885ba7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:25.059533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 598/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:25.083200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc324b5-d6c9-491d-95cc-1e24cfc904f9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:25.142736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:25.185458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc324b5-d6c9-491d-95cc-1e24cfc904f9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:25.219316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dc2a0-2b8e-4aee-b7c2-8c48a3b7a418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:25.220033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dc2a0-2b8e-4aee-b7c2-8c48a3b7a418 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:25.220314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dc2a0-2b8e-4aee-b7c2-8c48a3b7a418 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:25.220590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dc2a0-2b8e-4aee-b7c2-8c48a3b7a418 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:25.231969 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dc2a0-2b8e-4aee-b7c2-8c48a3b7a418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-482dc2a0-2b8e-4aee-b7c2-8c48a3b7a418 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:04:25.240338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:04:25.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:04:25.243158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 582/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:04:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:04:25.248387 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af192bf-2289-4c1b-8ab8-b6805885ba7d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:25.345690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af192bf-2289-4c1b-8ab8-b6805885ba7d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:25.347292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:25.379851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf618a-b082-467f-b28b-c4c5385d3bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:25.380507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf618a-b082-467f-b28b-c4c5385d3bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:25.380790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf618a-b082-467f-b28b-c4c5385d3bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:25.381065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf618a-b082-467f-b28b-c4c5385d3bb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:25.382451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf618a-b082-467f-b28b-c4c5385d3bb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:25.382754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf618a-b082-467f-b28b-c4c5385d3bb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:25.384118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf618a-b082-467f-b28b-c4c5385d3bb9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:25.392089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 599/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:04:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:25.397893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dc2a0-2b8e-4aee-b7c2-8c48a3b7a418 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:25.498137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dc2a0-2b8e-4aee-b7c2-8c48a3b7a418 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:25.515442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:25.531554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93e19d-39f1-49fc-949d-113c11b985fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:25.532417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93e19d-39f1-49fc-949d-113c11b985fe None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:25.532701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93e19d-39f1-49fc-949d-113c11b985fe None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:25.532978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93e19d-39f1-49fc-949d-113c11b985fe None admin] PATH_INFO: `/v3/users/900ed473f0f642549ef45ad37172c9c8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:25.551241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93e19d-39f1-49fc-949d-113c11b985fe None admin] RBAC: Authorizing `identity:get_user(user_id=900ed473f0f642549ef45ad37172c9c8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:25.552734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad93e19d-39f1-49fc-949d-113c11b985fe None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:25.555284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 583/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:04:25 2022] GET /identity/v3/users/900ed473f0f642549ef45ad37172c9c8 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:25.562712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf618a-b082-467f-b28b-c4c5385d3bb9 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:25.670304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf618a-b082-467f-b28b-c4c5385d3bb9 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:25.703503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79f334-dce3-44e4-a742-4830ecc2bf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:25.704146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79f334-dce3-44e4-a742-4830ecc2bf2b None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:25.704428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79f334-dce3-44e4-a742-4830ecc2bf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:25.704702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79f334-dce3-44e4-a742-4830ecc2bf2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:25.734997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:25.773482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79f334-dce3-44e4-a742-4830ecc2bf2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:25.774915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79f334-dce3-44e4-a742-4830ecc2bf2b None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:25.870492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 600/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:04:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:04:25.876380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93e19d-39f1-49fc-949d-113c11b985fe None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:25.985250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93e19d-39f1-49fc-949d-113c11b985fe None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:26.019548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591128be-16ad-4deb-b81a-c24b2cdc6755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:26.020500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591128be-16ad-4deb-b81a-c24b2cdc6755 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:26.020695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591128be-16ad-4deb-b81a-c24b2cdc6755 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:26.020968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591128be-16ad-4deb-b81a-c24b2cdc6755 None admin] PATH_INFO: `/v3/projects/329932a6f63a44d8b2818c9417b9b38e` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:26.022245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591128be-16ad-4deb-b81a-c24b2cdc6755 None admin] RBAC: Authorizing `identity:get_project(project_id=329932a6f63a44d8b2818c9417b9b38e)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:26.023868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591128be-16ad-4deb-b81a-c24b2cdc6755 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:26.025508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 584/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:04:25 2022] GET /identity/v3/projects/329932a6f63a44d8b2818c9417b9b38e => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:26.031557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79f334-dce3-44e4-a742-4830ecc2bf2b None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:26.093783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:26.130593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79f334-dce3-44e4-a742-4830ecc2bf2b None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:26.164799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409efeb5-9d14-44c0-97b2-013fc64a9459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:26.165439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409efeb5-9d14-44c0-97b2-013fc64a9459 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:26.165973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409efeb5-9d14-44c0-97b2-013fc64a9459 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:26.166249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409efeb5-9d14-44c0-97b2-013fc64a9459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:26.167095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409efeb5-9d14-44c0-97b2-013fc64a9459 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=329932a6f63a44d8b2818c9417b9b38e, user.id=900ed473f0f642549ef45ad37172c9c8) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:26.167407 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409efeb5-9d14-44c0-97b2-013fc64a9459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:26.168831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409efeb5-9d14-44c0-97b2-013fc64a9459 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:26.174137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 601/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:04:26 2022] GET /identity/v3/role_assignments?user.id=900ed473f0f642549ef45ad37172c9c8&scope.project.id=329932a6f63a44d8b2818c9417b9b38e&role.id=bd295cb5d1194b469beff0381c6b23af => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:28.378132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a6723-90f2-4fec-b068-bd76840fed4e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:28.378695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a6723-90f2-4fec-b068-bd76840fed4e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:28.378695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a6723-90f2-4fec-b068-bd76840fed4e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:28.380082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 585/1186] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:28.385865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aea361-6ef1-4fe6-86f0-e259ea6754e5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:28.386301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aea361-6ef1-4fe6-86f0-e259ea6754e5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:28.386471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aea361-6ef1-4fe6-86f0-e259ea6754e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:28.403143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-88aea361-6ef1-4fe6-86f0-e259ea6754e5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:28.404304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-88aea361-6ef1-4fe6-86f0-e259ea6754e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:28.469604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 602/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:28.493319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a6723-90f2-4fec-b068-bd76840fed4e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:28.539117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:28.593497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a6723-90f2-4fec-b068-bd76840fed4e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:28.628011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89ec03-9e7f-4431-8261-b47637ac6a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:28.628837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89ec03-9e7f-4431-8261-b47637ac6a85 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:28.629071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89ec03-9e7f-4431-8261-b47637ac6a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:28.629342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89ec03-9e7f-4431-8261-b47637ac6a85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:28.639250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89ec03-9e7f-4431-8261-b47637ac6a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-7f89ec03-9e7f-4431-8261-b47637ac6a85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:04:28.646891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:04:28.648381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:04:28.649493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 586/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:04:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:04:28.654307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aea361-6ef1-4fe6-86f0-e259ea6754e5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:28.711666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:28.751980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aea361-6ef1-4fe6-86f0-e259ea6754e5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:28.785915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701787cc-17e6-4109-8066-7b404c464fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:28.786615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701787cc-17e6-4109-8066-7b404c464fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:28.786907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701787cc-17e6-4109-8066-7b404c464fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:28.787186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701787cc-17e6-4109-8066-7b404c464fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:28.788640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701787cc-17e6-4109-8066-7b404c464fa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:28.788949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701787cc-17e6-4109-8066-7b404c464fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:28.790339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701787cc-17e6-4109-8066-7b404c464fa7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:28.798500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 603/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:04:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:28.804546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89ec03-9e7f-4431-8261-b47637ac6a85 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:28.903183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89ec03-9e7f-4431-8261-b47637ac6a85 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:28.905371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:28.936023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfb89b-784e-44e6-89b5-c18bc87e131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:28.936891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfb89b-784e-44e6-89b5-c18bc87e131b None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:28.937178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfb89b-784e-44e6-89b5-c18bc87e131b None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:28.937458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfb89b-784e-44e6-89b5-c18bc87e131b None admin] PATH_INFO: `/v3/users/900ed473f0f642549ef45ad37172c9c8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:28.938716 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfb89b-784e-44e6-89b5-c18bc87e131b None admin] RBAC: Authorizing `identity:get_user(user_id=900ed473f0f642549ef45ad37172c9c8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:28.940140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfb89b-784e-44e6-89b5-c18bc87e131b None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:28.941753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 587/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:04:28 2022] GET /identity/v3/users/900ed473f0f642549ef45ad37172c9c8 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:04:28.947488 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701787cc-17e6-4109-8066-7b404c464fa7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:29.047196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701787cc-17e6-4109-8066-7b404c464fa7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:29.066262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:29.079622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2befca-e936-44e4-9a4d-b9fa903d8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:29.080478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2befca-e936-44e4-9a4d-b9fa903d8e46 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:29.080759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2befca-e936-44e4-9a4d-b9fa903d8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:29.081032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2befca-e936-44e4-9a4d-b9fa903d8e46 None admin] PATH_INFO: `/v3/projects/329932a6f63a44d8b2818c9417b9b38e` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:29.082197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2befca-e936-44e4-9a4d-b9fa903d8e46 None admin] RBAC: Authorizing `identity:get_project(project_id=329932a6f63a44d8b2818c9417b9b38e)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:29.083679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2befca-e936-44e4-9a4d-b9fa903d8e46 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:29.085884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 604/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:04:28 2022] GET /identity/v3/projects/329932a6f63a44d8b2818c9417b9b38e => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:29.091810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfb89b-784e-44e6-89b5-c18bc87e131b None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:29.189139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfb89b-784e-44e6-89b5-c18bc87e131b None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:29.222013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff7b95-00b4-4ec1-953c-b471c7f53759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:29.222915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff7b95-00b4-4ec1-953c-b471c7f53759 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:29.223202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff7b95-00b4-4ec1-953c-b471c7f53759 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:29.223477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff7b95-00b4-4ec1-953c-b471c7f53759 None admin] PATH_INFO: `/v3/projects/329932a6f63a44d8b2818c9417b9b38e/users/900ed473f0f642549ef45ad37172c9c8/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:29.225596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff7b95-00b4-4ec1-953c-b471c7f53759 None admin] RBAC: Authorizing `identity:create_grant(project_id=329932a6f63a44d8b2818c9417b9b38e, user_id=900ed473f0f642549ef45ad37172c9c8, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:29.227137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff7b95-00b4-4ec1-953c-b471c7f53759 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:29.353443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 588/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:04:29 2022] PUT /identity/v3/projects/329932a6f63a44d8b2818c9417b9b38e/users/900ed473f0f642549ef45ad37172c9c8/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:04:31.566789 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd57ac3-8dd7-4e63-801f-7a595d897761 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:31.577556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd57ac3-8dd7-4e63-801f-7a595d897761 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:31.577556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd57ac3-8dd7-4e63-801f-7a595d897761 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:31.577556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 605/1193] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:04:31.577807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60a297-9f94-413f-91b4-71146d7c0246 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:31.577807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60a297-9f94-413f-91b4-71146d7c0246 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:31.577807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60a297-9f94-413f-91b4-71146d7c0246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:31.593174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7d60a297-9f94-413f-91b4-71146d7c0246 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:04:31.594353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7d60a297-9f94-413f-91b4-71146d7c0246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:04:31.619586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:31.661854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 589/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:04:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:04:31.686189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd57ac3-8dd7-4e63-801f-7a595d897761 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:31.785112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd57ac3-8dd7-4e63-801f-7a595d897761 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:31.818640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bff88-a8e9-4a75-aea1-2e475366c6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:31.819584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bff88-a8e9-4a75-aea1-2e475366c6fc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:31.819871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bff88-a8e9-4a75-aea1-2e475366c6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:31.820155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bff88-a8e9-4a75-aea1-2e475366c6fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:31.830102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bff88-a8e9-4a75-aea1-2e475366c6fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-f48bff88-a8e9-4a75-aea1-2e475366c6fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:04:31.838318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:04:31.839469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:04:31.840721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 606/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:04:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:04:31.845468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60a297-9f94-413f-91b4-71146d7c0246 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:31.943475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60a297-9f94-413f-91b4-71146d7c0246 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:31.977868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e687e6e-28e7-4a6e-9b4c-817fe95eb836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:31.978624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e687e6e-28e7-4a6e-9b4c-817fe95eb836 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:31.978905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e687e6e-28e7-4a6e-9b4c-817fe95eb836 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:31.979179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e687e6e-28e7-4a6e-9b4c-817fe95eb836 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:31.979909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e687e6e-28e7-4a6e-9b4c-817fe95eb836 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:31.980214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e687e6e-28e7-4a6e-9b4c-817fe95eb836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:31.981576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e687e6e-28e7-4a6e-9b4c-817fe95eb836 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:31.989407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 590/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:31.995787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bff88-a8e9-4a75-aea1-2e475366c6fc None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:32.044448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:32.094134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bff88-a8e9-4a75-aea1-2e475366c6fc None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:32.126694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310c351-c318-483a-a114-7ca9b6b327bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:32.127558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310c351-c318-483a-a114-7ca9b6b327bb None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:32.127838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310c351-c318-483a-a114-7ca9b6b327bb None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:32.128112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310c351-c318-483a-a114-7ca9b6b327bb None admin] PATH_INFO: `/v3/users/900ed473f0f642549ef45ad37172c9c8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:32.129282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310c351-c318-483a-a114-7ca9b6b327bb None admin] RBAC: Authorizing `identity:get_user(user_id=900ed473f0f642549ef45ad37172c9c8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:32.130748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310c351-c318-483a-a114-7ca9b6b327bb None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:32.132936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 607/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 19 12:04:31 2022] GET /identity/v3/users/900ed473f0f642549ef45ad37172c9c8 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:32.138684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e687e6e-28e7-4a6e-9b4c-817fe95eb836 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:32.164580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:32.237453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e687e6e-28e7-4a6e-9b4c-817fe95eb836 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:32.270191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83eacbb-1936-4dbc-bf2f-5a525ef1a807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:32.270855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83eacbb-1936-4dbc-bf2f-5a525ef1a807 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:32.271135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83eacbb-1936-4dbc-bf2f-5a525ef1a807 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:32.271508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83eacbb-1936-4dbc-bf2f-5a525ef1a807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:32.337799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83eacbb-1936-4dbc-bf2f-5a525ef1a807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:32.339264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83eacbb-1936-4dbc-bf2f-5a525ef1a807 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:32.361707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:32.433867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 591/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:04:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:04:32.439760 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310c351-c318-483a-a114-7ca9b6b327bb None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:32.538390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310c351-c318-483a-a114-7ca9b6b327bb None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:32.555159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:32.572984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640e07f-ecf1-44d2-a6ce-b73a036c85dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:32.573847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640e07f-ecf1-44d2-a6ce-b73a036c85dc None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:32.574138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640e07f-ecf1-44d2-a6ce-b73a036c85dc None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:32.574433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640e07f-ecf1-44d2-a6ce-b73a036c85dc None admin] PATH_INFO: `/v3/projects/329932a6f63a44d8b2818c9417b9b38e` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:32.575624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640e07f-ecf1-44d2-a6ce-b73a036c85dc None admin] RBAC: Authorizing `identity:get_project(project_id=329932a6f63a44d8b2818c9417b9b38e)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:32.577050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640e07f-ecf1-44d2-a6ce-b73a036c85dc None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:32.578764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 608/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 19 12:04:32 2022] GET /identity/v3/projects/329932a6f63a44d8b2818c9417b9b38e => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:32.585084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83eacbb-1936-4dbc-bf2f-5a525ef1a807 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:04:32.683046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83eacbb-1936-4dbc-bf2f-5a525ef1a807 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:04:32.705423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:32.716356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5675ca-b836-4294-a073-e288843900d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:04:32.719087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5675ca-b836-4294-a073-e288843900d6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:04:32.719087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5675ca-b836-4294-a073-e288843900d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:04:32.719087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5675ca-b836-4294-a073-e288843900d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:04:32.719087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5675ca-b836-4294-a073-e288843900d6 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=329932a6f63a44d8b2818c9417b9b38e, user.id=900ed473f0f642549ef45ad37172c9c8) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:04:32.719087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5675ca-b836-4294-a073-e288843900d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:04:32.720224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5675ca-b836-4294-a073-e288843900d6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:04:32.726865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 592/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:04:32 2022] GET /identity/v3/role_assignments?user.id=900ed473f0f642549ef45ad37172c9c8&scope.project.id=329932a6f63a44d8b2818c9417b9b38e&role.id=bd295cb5d1194b469beff0381c6b23af => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:04:42.593628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=8 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:04:42.753994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:00.271454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a63e2-f7aa-4b0e-b592-84a5cd14441d None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:00.272128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a63e2-f7aa-4b0e-b592-84a5cd14441d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:00.272128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a63e2-f7aa-4b0e-b592-84a5cd14441d None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:00.273404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 609/1201] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:00.279590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777507c7-c85d-4a7f-9953-ec810e2f686f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:00.284822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777507c7-c85d-4a7f-9953-ec810e2f686f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:00.284822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777507c7-c85d-4a7f-9953-ec810e2f686f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:00.297797 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-777507c7-c85d-4a7f-9953-ec810e2f686f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:00.299019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-777507c7-c85d-4a7f-9953-ec810e2f686f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:00.366358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 593/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:00.392189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a63e2-f7aa-4b0e-b592-84a5cd14441d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:00.494002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a63e2-f7aa-4b0e-b592-84a5cd14441d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:00.528901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccc005-9c1e-4338-95ed-833b2af002b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:00.529625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccc005-9c1e-4338-95ed-833b2af002b1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:00.529910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccc005-9c1e-4338-95ed-833b2af002b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:00.530186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccc005-9c1e-4338-95ed-833b2af002b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:00.544832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccc005-9c1e-4338-95ed-833b2af002b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:00.546900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccc005-9c1e-4338-95ed-833b2af002b1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a7ccc005-9c1e-4338-95ed-833b2af002b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:00.568759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:00.569885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:00.570960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:00.570960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:00.570960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:00.570960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:00.570960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 610/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:00.576593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777507c7-c85d-4a7f-9953-ec810e2f686f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:00.681890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777507c7-c85d-4a7f-9953-ec810e2f686f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:00.710001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:00.716115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d38c8d-79da-40c5-8a14-06b9cfcd03fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:00.716766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d38c8d-79da-40c5-8a14-06b9cfcd03fd None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:00.717103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d38c8d-79da-40c5-8a14-06b9cfcd03fd None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:00.717405 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d38c8d-79da-40c5-8a14-06b9cfcd03fd None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:00.718114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d38c8d-79da-40c5-8a14-06b9cfcd03fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:00.718479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d38c8d-79da-40c5-8a14-06b9cfcd03fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:00.719834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d38c8d-79da-40c5-8a14-06b9cfcd03fd None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:00.731977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 594/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:00.737662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccc005-9c1e-4338-95ed-833b2af002b1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:00.789786 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:00.836999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccc005-9c1e-4338-95ed-833b2af002b1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:00.871988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17f24e-7928-41dc-8956-fa850857940e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:00.872657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17f24e-7928-41dc-8956-fa850857940e None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:00.872942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17f24e-7928-41dc-8956-fa850857940e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:00.873223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17f24e-7928-41dc-8956-fa850857940e None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:00.873925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17f24e-7928-41dc-8956-fa850857940e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:00.875396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17f24e-7928-41dc-8956-fa850857940e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:00.988139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 611/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 19 12:06:00 2022] POST /identity/v3/users => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:06:03.269020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75543e4-6576-4be9-bef2-b136f9302016 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:03.269583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75543e4-6576-4be9-bef2-b136f9302016 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:03.269583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75543e4-6576-4be9-bef2-b136f9302016 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:03.270969 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 595/1206] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:03.276905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0532cd-b478-4c92-bf21-0867beb7d7bd None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:03.277354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0532cd-b478-4c92-bf21-0867beb7d7bd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:03.277501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0532cd-b478-4c92-bf21-0867beb7d7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:03.294921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2e0532cd-b478-4c92-bf21-0867beb7d7bd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:03.296116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2e0532cd-b478-4c92-bf21-0867beb7d7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:03.361282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 612/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:03.384682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75543e4-6576-4be9-bef2-b136f9302016 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:03.486747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75543e4-6576-4be9-bef2-b136f9302016 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:03.521479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8322053-7124-4a92-8c21-a61a70376147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:03.524931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8322053-7124-4a92-8c21-a61a70376147 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:03.524931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8322053-7124-4a92-8c21-a61a70376147 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:03.524931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8322053-7124-4a92-8c21-a61a70376147 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:03.533970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8322053-7124-4a92-8c21-a61a70376147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-d8322053-7124-4a92-8c21-a61a70376147 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:06:03.541713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:03.543135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:03.544086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 596/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:06:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:03.549248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0532cd-b478-4c92-bf21-0867beb7d7bd None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:03.588218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:03.653530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0532cd-b478-4c92-bf21-0867beb7d7bd None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:03.687427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1d51e-0cc1-4459-aaa5-ab12ff32458e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:03.688087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1d51e-0cc1-4459-aaa5-ab12ff32458e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:03.688366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1d51e-0cc1-4459-aaa5-ab12ff32458e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:03.688637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1d51e-0cc1-4459-aaa5-ab12ff32458e None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:03.689367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1d51e-0cc1-4459-aaa5-ab12ff32458e None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:03.689743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1d51e-0cc1-4459-aaa5-ab12ff32458e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:03.691225 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1d51e-0cc1-4459-aaa5-ab12ff32458e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:03.699803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 613/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:06:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:06:03.705599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8322053-7124-4a92-8c21-a61a70376147 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:03.755368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:03.804573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8322053-7124-4a92-8c21-a61a70376147 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:03.843382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1867fa-470e-42d2-a46f-31e91898109e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:03.844248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1867fa-470e-42d2-a46f-31e91898109e None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:03.844533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1867fa-470e-42d2-a46f-31e91898109e None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:03.844808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1867fa-470e-42d2-a46f-31e91898109e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:03.855718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1867fa-470e-42d2-a46f-31e91898109e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:03.857165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1867fa-470e-42d2-a46f-31e91898109e None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-bc1867fa-470e-42d2-a46f-31e91898109e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:03.874948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:03.886676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:03.887763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:03.887763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:03.887763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:03.887763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:03.887763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 597/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:03.888109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1d51e-0cc1-4459-aaa5-ab12ff32458e None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:03.981400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:03.984488 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1d51e-0cc1-4459-aaa5-ab12ff32458e None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:04.018569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b44539-5354-48ef-bf56-e25a907e2274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:04.019237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44539-5354-48ef-bf56-e25a907e2274 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:04.019522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44539-5354-48ef-bf56-e25a907e2274 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:04.019799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b44539-5354-48ef-bf56-e25a907e2274 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:04.022813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b44539-5354-48ef-bf56-e25a907e2274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:04.022813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b44539-5354-48ef-bf56-e25a907e2274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:04.022813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b44539-5354-48ef-bf56-e25a907e2274 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:04.034213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 614/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:04.040034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1867fa-470e-42d2-a46f-31e91898109e None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:04.140659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1867fa-470e-42d2-a46f-31e91898109e None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:04.142199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:04.175079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09732a-e3d6-428b-84cb-8c7978a82b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:04.175884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09732a-e3d6-428b-84cb-8c7978a82b10 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:04.176168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09732a-e3d6-428b-84cb-8c7978a82b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:04.176462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09732a-e3d6-428b-84cb-8c7978a82b10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:04.184153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09732a-e3d6-428b-84cb-8c7978a82b10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:04.185613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c09732a-e3d6-428b-84cb-8c7978a82b10 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-1c09732a-e3d6-428b-84cb-8c7978a82b10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:06:04.200723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:04.201885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:04.203418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 598/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:04.207799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b44539-5354-48ef-bf56-e25a907e2274 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:04.308473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b44539-5354-48ef-bf56-e25a907e2274 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:04.340311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6874a4-9486-4377-8f78-d7c107fc8445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:04.340965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6874a4-9486-4377-8f78-d7c107fc8445 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:04.341243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6874a4-9486-4377-8f78-d7c107fc8445 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:04.341511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6874a4-9486-4377-8f78-d7c107fc8445 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:04.342396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6874a4-9486-4377-8f78-d7c107fc8445 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:04.342715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6874a4-9486-4377-8f78-d7c107fc8445 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:04.344229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6874a4-9486-4377-8f78-d7c107fc8445 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:04.362190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 615/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 12:06:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:04.367925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09732a-e3d6-428b-84cb-8c7978a82b10 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:04.469053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09732a-e3d6-428b-84cb-8c7978a82b10 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:04.503336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a6e49-c9a2-4756-bc02-13843773780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:04.503978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a6e49-c9a2-4756-bc02-13843773780c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:04.504262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a6e49-c9a2-4756-bc02-13843773780c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:04.504539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a6e49-c9a2-4756-bc02-13843773780c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:04.534188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:04.572695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a6e49-c9a2-4756-bc02-13843773780c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:04.574303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a6e49-c9a2-4756-bc02-13843773780c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:04.675971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 599/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:06:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:06:04.681773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6874a4-9486-4377-8f78-d7c107fc8445 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:04.701086 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:04.783252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6874a4-9486-4377-8f78-d7c107fc8445 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:04.817439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a47a1-b466-45e9-a654-55a19a4dda82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:04.818222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a47a1-b466-45e9-a654-55a19a4dda82 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:04.818527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a47a1-b466-45e9-a654-55a19a4dda82 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:04.818798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a47a1-b466-45e9-a654-55a19a4dda82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:04.829187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a47a1-b466-45e9-a654-55a19a4dda82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:04.830658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a47a1-b466-45e9-a654-55a19a4dda82 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e50a47a1-b466-45e9-a654-55a19a4dda82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:04.848863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:04.857859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:04.858847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:04.858847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:04.858847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:04.858847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:04.858847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 616/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:04.859142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a6e49-c9a2-4756-bc02-13843773780c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:04.915721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:04.952585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a6e49-c9a2-4756-bc02-13843773780c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:04.988383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e9d85-76e5-4311-85a7-996cc3c118ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:04.989042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e9d85-76e5-4311-85a7-996cc3c118ed None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:04.989326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e9d85-76e5-4311-85a7-996cc3c118ed None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:04.989603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e9d85-76e5-4311-85a7-996cc3c118ed None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:04.990314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e9d85-76e5-4311-85a7-996cc3c118ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:04.990662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e9d85-76e5-4311-85a7-996cc3c118ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:04.991997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e9d85-76e5-4311-85a7-996cc3c118ed None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:05.003058 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 600/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:05.008647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a47a1-b466-45e9-a654-55a19a4dda82 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:05.079114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:05.109499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a47a1-b466-45e9-a654-55a19a4dda82 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:05.142260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7e58-a054-4401-9924-cb7a44b9e20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:05.143076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7e58-a054-4401-9924-cb7a44b9e20c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:05.143357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7e58-a054-4401-9924-cb7a44b9e20c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:05.143634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7e58-a054-4401-9924-cb7a44b9e20c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:05.150031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed7e58-a054-4401-9924-cb7a44b9e20c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:05.151671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed7e58-a054-4401-9924-cb7a44b9e20c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-04ed7e58-a054-4401-9924-cb7a44b9e20c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:06:05.164926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:05.165919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:05.167859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:06:05.167859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:06:05.167859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:06:05.167859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:06:05.167859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:06:05.167859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:06:05.167859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:05.167859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:05.167859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 617/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:06:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:05.172826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e9d85-76e5-4311-85a7-996cc3c118ed None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:05.273302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e9d85-76e5-4311-85a7-996cc3c118ed None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:05.296340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:05.308396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82686e03-525e-415f-a6dc-e3ea1639a48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:05.309053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82686e03-525e-415f-a6dc-e3ea1639a48d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:05.309337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82686e03-525e-415f-a6dc-e3ea1639a48d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:05.309613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82686e03-525e-415f-a6dc-e3ea1639a48d None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:05.310390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82686e03-525e-415f-a6dc-e3ea1639a48d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:05.310730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82686e03-525e-415f-a6dc-e3ea1639a48d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:05.312147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82686e03-525e-415f-a6dc-e3ea1639a48d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:05.324114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 601/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:06:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:06:05.330278 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7e58-a054-4401-9924-cb7a44b9e20c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:05.429813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7e58-a054-4401-9924-cb7a44b9e20c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:05.456747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:05.462442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163726a-5ebf-4865-9d9b-f96f71fda16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:05.463090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163726a-5ebf-4865-9d9b-f96f71fda16e None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:05.463932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163726a-5ebf-4865-9d9b-f96f71fda16e None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:05.464266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163726a-5ebf-4865-9d9b-f96f71fda16e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:05.465095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163726a-5ebf-4865-9d9b-f96f71fda16e None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=4f41920bad3147bba0000de911dfe5a1) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:05.465405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163726a-5ebf-4865-9d9b-f96f71fda16e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:05.466897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163726a-5ebf-4865-9d9b-f96f71fda16e None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:05.472414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 618/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:06:05 2022] GET /identity/v3/role_assignments?user.id=4f41920bad3147bba0000de911dfe5a1&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:07.735855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdffd0-f1b4-4a96-a3f6-ab810875a4ee None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:07.736386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdffd0-f1b4-4a96-a3f6-ab810875a4ee None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:07.736386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdffd0-f1b4-4a96-a3f6-ab810875a4ee None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:07.737749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 602/1220] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:07.742868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce689-fd45-4937-8e72-7bd7d6cd83d4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:07.743319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce689-fd45-4937-8e72-7bd7d6cd83d4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:07.743444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ce689-fd45-4937-8e72-7bd7d6cd83d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:07.760289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-119ce689-fd45-4937-8e72-7bd7d6cd83d4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:07.761621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-119ce689-fd45-4937-8e72-7bd7d6cd83d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:07.827714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 619/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:07.850505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdffd0-f1b4-4a96-a3f6-ab810875a4ee None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:07.950918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdffd0-f1b4-4a96-a3f6-ab810875a4ee None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:07.984386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1619c3-8a7b-460d-b559-78cc45a51fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:07.985170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1619c3-8a7b-460d-b559-78cc45a51fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:07.985456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1619c3-8a7b-460d-b559-78cc45a51fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:07.985732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1619c3-8a7b-460d-b559-78cc45a51fc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:07.995717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1619c3-8a7b-460d-b559-78cc45a51fc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-3a1619c3-8a7b-460d-b559-78cc45a51fc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:06:08.003395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:08.004391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:08.005235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 603/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:06:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:08.011084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce689-fd45-4937-8e72-7bd7d6cd83d4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:08.108650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ce689-fd45-4937-8e72-7bd7d6cd83d4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:08.131217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:08.141953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a240a-4b59-4c88-a224-41408d2fbef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:08.142651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a240a-4b59-4c88-a224-41408d2fbef1 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:08.142937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a240a-4b59-4c88-a224-41408d2fbef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:08.143214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a240a-4b59-4c88-a224-41408d2fbef1 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:08.143966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a240a-4b59-4c88-a224-41408d2fbef1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:08.144263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a240a-4b59-4c88-a224-41408d2fbef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:08.145642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a240a-4b59-4c88-a224-41408d2fbef1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:08.155706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 620/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:06:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:08.160608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1619c3-8a7b-460d-b559-78cc45a51fc9 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:08.206005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:08.263077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1619c3-8a7b-460d-b559-78cc45a51fc9 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:08.297088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fdc60-9935-47d8-ad94-4374859cdfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:08.297878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fdc60-9935-47d8-ad94-4374859cdfdb None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:08.298155 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fdc60-9935-47d8-ad94-4374859cdfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:08.298545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fdc60-9935-47d8-ad94-4374859cdfdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:08.308909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fdc60-9935-47d8-ad94-4374859cdfdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:08.310358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fdc60-9935-47d8-ad94-4374859cdfdb None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-f36fdc60-9935-47d8-ad94-4374859cdfdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:08.327480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:08.333887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:08.334873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:08.334873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:08.334873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:08.334873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:08.334873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 604/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:08.335161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a240a-4b59-4c88-a224-41408d2fbef1 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:08.436440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a240a-4b59-4c88-a224-41408d2fbef1 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:08.470160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c01fb-239e-4d4e-a48b-f0ea7e624370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:08.470942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c01fb-239e-4d4e-a48b-f0ea7e624370 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:08.471212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c01fb-239e-4d4e-a48b-f0ea7e624370 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:08.471485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c01fb-239e-4d4e-a48b-f0ea7e624370 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:08.472209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c01fb-239e-4d4e-a48b-f0ea7e624370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:08.472514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c01fb-239e-4d4e-a48b-f0ea7e624370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:08.473860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c01fb-239e-4d4e-a48b-f0ea7e624370 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:08.485833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 621/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:08.491666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fdc60-9935-47d8-ad94-4374859cdfdb None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:08.587579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:08.594386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fdc60-9935-47d8-ad94-4374859cdfdb None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:08.627098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f47d83-430a-4e57-9426-a35ce3eb5831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:08.627922 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f47d83-430a-4e57-9426-a35ce3eb5831 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:08.628207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f47d83-430a-4e57-9426-a35ce3eb5831 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:08.628483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f47d83-430a-4e57-9426-a35ce3eb5831 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:08.636390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f47d83-430a-4e57-9426-a35ce3eb5831 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:08.637830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f47d83-430a-4e57-9426-a35ce3eb5831 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-76f47d83-430a-4e57-9426-a35ce3eb5831 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:06:08.651976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:08.652910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:08.654003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 605/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:08.659195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c01fb-239e-4d4e-a48b-f0ea7e624370 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:08.708974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:08.760068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c01fb-239e-4d4e-a48b-f0ea7e624370 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:08.795056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34f358-0c74-4e99-8751-95df2f0c3b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:08.799115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34f358-0c74-4e99-8751-95df2f0c3b1a None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:08.799115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34f358-0c74-4e99-8751-95df2f0c3b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:08.799115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34f358-0c74-4e99-8751-95df2f0c3b1a None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:08.799115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34f358-0c74-4e99-8751-95df2f0c3b1a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:08.799115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34f358-0c74-4e99-8751-95df2f0c3b1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:08.799115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34f358-0c74-4e99-8751-95df2f0c3b1a None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:08.816121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 622/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 12:06:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:08.821879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f47d83-430a-4e57-9426-a35ce3eb5831 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:08.924373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f47d83-430a-4e57-9426-a35ce3eb5831 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:08.957401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f311cf-4f61-479b-aabf-8e4b2a85c4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:08.958179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f311cf-4f61-479b-aabf-8e4b2a85c4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:08.958485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f311cf-4f61-479b-aabf-8e4b2a85c4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:08.958759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f311cf-4f61-479b-aabf-8e4b2a85c4c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:08.969555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f311cf-4f61-479b-aabf-8e4b2a85c4c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:08.971048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f311cf-4f61-479b-aabf-8e4b2a85c4c6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-05f311cf-4f61-479b-aabf-8e4b2a85c4c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:08.988701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:08.989738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:08.991045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:08.991045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:08.991045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:08.991045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:08.991045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 606/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:08.996395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34f358-0c74-4e99-8751-95df2f0c3b1a None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:09.096162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34f358-0c74-4e99-8751-95df2f0c3b1a None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:09.098496 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:09.130484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd2daf-01a1-4528-b646-4eaef60e530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:09.131132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd2daf-01a1-4528-b646-4eaef60e530c None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:09.131416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd2daf-01a1-4528-b646-4eaef60e530c None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:09.131691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd2daf-01a1-4528-b646-4eaef60e530c None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:09.132399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd2daf-01a1-4528-b646-4eaef60e530c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:09.132703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd2daf-01a1-4528-b646-4eaef60e530c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:09.134055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd2daf-01a1-4528-b646-4eaef60e530c None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:09.145094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 623/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:06:09.151736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f311cf-4f61-479b-aabf-8e4b2a85c4c6 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:09.161276 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:09.257888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f311cf-4f61-479b-aabf-8e4b2a85c4c6 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:09.290898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e506-73b2-4964-aa6b-7dd880dc0d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:09.293232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e506-73b2-4964-aa6b-7dd880dc0d93 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:09.293232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e506-73b2-4964-aa6b-7dd880dc0d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:09.293232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e506-73b2-4964-aa6b-7dd880dc0d93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:09.299593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19e506-73b2-4964-aa6b-7dd880dc0d93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:09.301030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19e506-73b2-4964-aa6b-7dd880dc0d93 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-9a19e506-73b2-4964-aa6b-7dd880dc0d93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:06:09.315060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:09.316099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:09.317204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:06:09.317204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:06:09.317204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:06:09.317204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:06:09.317204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:06:09.317204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:06:09.317204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:09.317204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:09.317204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 607/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:09.322034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd2daf-01a1-4528-b646-4eaef60e530c None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:09.420357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd2daf-01a1-4528-b646-4eaef60e530c None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:09.452757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58a925-de70-473d-90f2-d8bf94041064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:09.453395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58a925-de70-473d-90f2-d8bf94041064 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:09.453708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58a925-de70-473d-90f2-d8bf94041064 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:09.453952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58a925-de70-473d-90f2-d8bf94041064 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:09.454746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58a925-de70-473d-90f2-d8bf94041064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:09.455079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58a925-de70-473d-90f2-d8bf94041064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:09.456502 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58a925-de70-473d-90f2-d8bf94041064 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:09.469328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 624/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:06:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:09.475513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e506-73b2-4964-aa6b-7dd880dc0d93 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:09.542954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:09.581931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e506-73b2-4964-aa6b-7dd880dc0d93 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:09.615888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd43e4-8f78-4b1e-9f70-a6f44a9a5de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:09.616759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd43e4-8f78-4b1e-9f70-a6f44a9a5de0 None admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:09.617038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd43e4-8f78-4b1e-9f70-a6f44a9a5de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:09.617309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd43e4-8f78-4b1e-9f70-a6f44a9a5de0 None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/4f41920bad3147bba0000de911dfe5a1/roles/941f412208c04a7abd395240a0339304` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:09.631545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd43e4-8f78-4b1e-9f70-a6f44a9a5de0 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=4f41920bad3147bba0000de911dfe5a1, role_id=941f412208c04a7abd395240a0339304)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:09.632989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd43e4-8f78-4b1e-9f70-a6f44a9a5de0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:09.716638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 608/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:06:09 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/4f41920bad3147bba0000de911dfe5a1/roles/941f412208c04a7abd395240a0339304 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:06:11.912797 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f60fa4-74f5-4c4e-98b9-4ecc80afeb5f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:11.913354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f60fa4-74f5-4c4e-98b9-4ecc80afeb5f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:11.913354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f60fa4-74f5-4c4e-98b9-4ecc80afeb5f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:11.914754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 625/1233] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:11.920651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdcda3-19b7-4506-a47c-7777c888b7cf None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:11.921053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdcda3-19b7-4506-a47c-7777c888b7cf None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:11.921207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdcda3-19b7-4506-a47c-7777c888b7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:11.937773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-47cdcda3-19b7-4506-a47c-7777c888b7cf None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:11.938999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-47cdcda3-19b7-4506-a47c-7777c888b7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:12.005020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 609/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:12.028132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f60fa4-74f5-4c4e-98b9-4ecc80afeb5f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:12.054193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:12.134185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f60fa4-74f5-4c4e-98b9-4ecc80afeb5f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:12.169068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09f977-48a2-4543-adef-ae471c1f27ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:12.169787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09f977-48a2-4543-adef-ae471c1f27ec None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:12.170071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09f977-48a2-4543-adef-ae471c1f27ec None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:12.170351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09f977-48a2-4543-adef-ae471c1f27ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:12.182123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09f977-48a2-4543-adef-ae471c1f27ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-ee09f977-48a2-4543-adef-ae471c1f27ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:06:12.189913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:12.190994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:12.191863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 626/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 19 12:06:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:12.197141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdcda3-19b7-4506-a47c-7777c888b7cf None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:12.243584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:12.295618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdcda3-19b7-4506-a47c-7777c888b7cf None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:12.329216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697796fd-4fda-4cc1-90ff-b07e2c6d78f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:12.329916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697796fd-4fda-4cc1-90ff-b07e2c6d78f4 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:12.330200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697796fd-4fda-4cc1-90ff-b07e2c6d78f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:12.335033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697796fd-4fda-4cc1-90ff-b07e2c6d78f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:12.335033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697796fd-4fda-4cc1-90ff-b07e2c6d78f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:12.335033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697796fd-4fda-4cc1-90ff-b07e2c6d78f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:12.335033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697796fd-4fda-4cc1-90ff-b07e2c6d78f4 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:12.342185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 610/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 19 12:06:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:12.348087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09f977-48a2-4543-adef-ae471c1f27ec None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:12.431937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:12.450742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09f977-48a2-4543-adef-ae471c1f27ec None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:12.486493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6ecdd-dbeb-4af8-852d-d04ef8590703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:12.490875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6ecdd-dbeb-4af8-852d-d04ef8590703 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:12.490875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6ecdd-dbeb-4af8-852d-d04ef8590703 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:12.490875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6ecdd-dbeb-4af8-852d-d04ef8590703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:12.498009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6ecdd-dbeb-4af8-852d-d04ef8590703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:12.499486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6ecdd-dbeb-4af8-852d-d04ef8590703 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-afb6ecdd-dbeb-4af8-852d-d04ef8590703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:12.516744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:12.517794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:12.519074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:12.519074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:12.519074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:12.519074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:12.519074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 627/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:12.523722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697796fd-4fda-4cc1-90ff-b07e2c6d78f4 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:12.625224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697796fd-4fda-4cc1-90ff-b07e2c6d78f4 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:12.634539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:12.659993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a32d6-1f47-4b45-a2cd-f3560b12bba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:12.661123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a32d6-1f47-4b45-a2cd-f3560b12bba5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:12.661599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a32d6-1f47-4b45-a2cd-f3560b12bba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:12.662072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a32d6-1f47-4b45-a2cd-f3560b12bba5 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:12.663435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a32d6-1f47-4b45-a2cd-f3560b12bba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:12.664005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a32d6-1f47-4b45-a2cd-f3560b12bba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:12.673964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a32d6-1f47-4b45-a2cd-f3560b12bba5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:12.680664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 611/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:12.686741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6ecdd-dbeb-4af8-852d-d04ef8590703 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:12.792672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6ecdd-dbeb-4af8-852d-d04ef8590703 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:12.830236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ca010-b6c6-4b7e-9379-98017f047323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:12.831054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ca010-b6c6-4b7e-9379-98017f047323 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:12.831334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ca010-b6c6-4b7e-9379-98017f047323 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:12.831632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ca010-b6c6-4b7e-9379-98017f047323 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:12.834307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:12.839830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ca010-b6c6-4b7e-9379-98017f047323 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:12.841260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ca010-b6c6-4b7e-9379-98017f047323 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-009ca010-b6c6-4b7e-9379-98017f047323 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:06:12.855862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:12.856961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:12.858260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 628/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:12.862847 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a32d6-1f47-4b45-a2cd-f3560b12bba5 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:12.965936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a32d6-1f47-4b45-a2cd-f3560b12bba5 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:13.000780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceb885-2c3e-4e8c-95ee-c0058613e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:13.001448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceb885-2c3e-4e8c-95ee-c0058613e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:13.001731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceb885-2c3e-4e8c-95ee-c0058613e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:13.002013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceb885-2c3e-4e8c-95ee-c0058613e0f0 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:13.002874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ceb885-2c3e-4e8c-95ee-c0058613e0f0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:13.003186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ceb885-2c3e-4e8c-95ee-c0058613e0f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:13.004612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ceb885-2c3e-4e8c-95ee-c0058613e0f0 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:13.024316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 612/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 12:06:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:13.029553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ca010-b6c6-4b7e-9379-98017f047323 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:13.127157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ca010-b6c6-4b7e-9379-98017f047323 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:13.160388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec67bfb-90e1-49fe-bc6c-be79cf106939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:13.161031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67bfb-90e1-49fe-bc6c-be79cf106939 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:13.161313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67bfb-90e1-49fe-bc6c-be79cf106939 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:13.161589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec67bfb-90e1-49fe-bc6c-be79cf106939 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:13.221689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:13.227974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec67bfb-90e1-49fe-bc6c-be79cf106939 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:13.229566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec67bfb-90e1-49fe-bc6c-be79cf106939 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:13.332404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 629/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:06:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:06:13.338497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceb885-2c3e-4e8c-95ee-c0058613e0f0 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:13.366461 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:13.438631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceb885-2c3e-4e8c-95ee-c0058613e0f0 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:13.471139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86064d-b6eb-4b2b-9149-772fb1ade97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:13.471987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86064d-b6eb-4b2b-9149-772fb1ade97d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:13.472210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86064d-b6eb-4b2b-9149-772fb1ade97d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:13.472482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86064d-b6eb-4b2b-9149-772fb1ade97d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:13.483494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86064d-b6eb-4b2b-9149-772fb1ade97d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:13.484990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86064d-b6eb-4b2b-9149-772fb1ade97d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-4d86064d-b6eb-4b2b-9149-772fb1ade97d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:13.502550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:13.503509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:13.504416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:13.504416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:13.504416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:13.504416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:13.504416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 613/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:13.509544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec67bfb-90e1-49fe-bc6c-be79cf106939 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:13.602427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:13.608698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec67bfb-90e1-49fe-bc6c-be79cf106939 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:13.642165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1bd6-2a7a-4f4f-94a8-e7abfbeb2b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:13.642857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1bd6-2a7a-4f4f-94a8-e7abfbeb2b44 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:13.643138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1bd6-2a7a-4f4f-94a8-e7abfbeb2b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:13.643411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f1bd6-2a7a-4f4f-94a8-e7abfbeb2b44 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:13.644122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f1bd6-2a7a-4f4f-94a8-e7abfbeb2b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:13.644428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f1bd6-2a7a-4f4f-94a8-e7abfbeb2b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:13.645835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f1bd6-2a7a-4f4f-94a8-e7abfbeb2b44 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:13.657963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 630/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:13.663647 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86064d-b6eb-4b2b-9149-772fb1ade97d None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:13.744568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:13.765430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86064d-b6eb-4b2b-9149-772fb1ade97d None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:13.799796 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679162ab-489d-4a51-ba65-546a1a81b52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:13.800632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679162ab-489d-4a51-ba65-546a1a81b52c None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:13.800916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679162ab-489d-4a51-ba65-546a1a81b52c None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:13.801192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679162ab-489d-4a51-ba65-546a1a81b52c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:13.807812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679162ab-489d-4a51-ba65-546a1a81b52c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:13.809234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679162ab-489d-4a51-ba65-546a1a81b52c None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-679162ab-489d-4a51-ba65-546a1a81b52c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:06:13.822989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:13.824063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:13.825306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:06:13.825306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:06:13.825306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:06:13.825306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:06:13.825306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:06:13.825306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:06:13.825306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:13.825306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:13.825306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 614/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:13.830337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1bd6-2a7a-4f4f-94a8-e7abfbeb2b44 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:13.929405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f1bd6-2a7a-4f4f-94a8-e7abfbeb2b44 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:13.962021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4233957d-5410-455b-834a-ccf275e1f1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:13.962691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233957d-5410-455b-834a-ccf275e1f1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:13.962973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233957d-5410-455b-834a-ccf275e1f1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:13.964191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4233957d-5410-455b-834a-ccf275e1f1e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:13.964975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233957d-5410-455b-834a-ccf275e1f1e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:13.965288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233957d-5410-455b-834a-ccf275e1f1e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:13.966729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4233957d-5410-455b-834a-ccf275e1f1e9 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:13.978287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 631/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:06:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:13.985008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679162ab-489d-4a51-ba65-546a1a81b52c None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:14.084511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679162ab-489d-4a51-ba65-546a1a81b52c None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:14.117202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc06d6-131f-4f42-8a97-49fcb5623c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:14.117843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc06d6-131f-4f42-8a97-49fcb5623c43 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:14.118124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc06d6-131f-4f42-8a97-49fcb5623c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:14.118428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc06d6-131f-4f42-8a97-49fcb5623c43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:14.119243 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc06d6-131f-4f42-8a97-49fcb5623c43 None admin] RBAC: Adding query filter params (role.id=941f412208c04a7abd395240a0339304, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=4f41920bad3147bba0000de911dfe5a1) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:14.120329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc06d6-131f-4f42-8a97-49fcb5623c43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:14.121792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc06d6-131f-4f42-8a97-49fcb5623c43 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:14.127718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 615/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:06:13 2022] GET /identity/v3/role_assignments?user.id=4f41920bad3147bba0000de911dfe5a1&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=941f412208c04a7abd395240a0339304 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:16.427019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22117d09-03fe-45c4-b894-eec54cd5a2be None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:16.427573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22117d09-03fe-45c4-b894-eec54cd5a2be None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:16.427573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22117d09-03fe-45c4-b894-eec54cd5a2be None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:16.428982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 632/1247] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:16.434920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced8396-3a6d-43eb-a871-3ae4fb6ad360 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:16.435390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced8396-3a6d-43eb-a871-3ae4fb6ad360 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:16.435502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced8396-3a6d-43eb-a871-3ae4fb6ad360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:16.440548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:16.452622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-dced8396-3a6d-43eb-a871-3ae4fb6ad360 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:16.453803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dced8396-3a6d-43eb-a871-3ae4fb6ad360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:16.518853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 616/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:16.544020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22117d09-03fe-45c4-b894-eec54cd5a2be None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:16.562721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:16.642460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22117d09-03fe-45c4-b894-eec54cd5a2be None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:16.676025 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495765af-c2b5-46f4-857d-afa832008c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:16.676829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495765af-c2b5-46f4-857d-afa832008c85 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:16.677171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495765af-c2b5-46f4-857d-afa832008c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:16.677394 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495765af-c2b5-46f4-857d-afa832008c85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:16.688893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495765af-c2b5-46f4-857d-afa832008c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-495765af-c2b5-46f4-857d-afa832008c85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:06:16.696599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:16.698254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:16.699132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 633/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:06:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:06:16.704773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced8396-3a6d-43eb-a871-3ae4fb6ad360 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:16.804923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced8396-3a6d-43eb-a871-3ae4fb6ad360 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:16.835664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:16.840533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266b98-b754-4ac5-a53b-86857c3eec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:16.841190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266b98-b754-4ac5-a53b-86857c3eec13 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:16.841479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266b98-b754-4ac5-a53b-86857c3eec13 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:16.841757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266b98-b754-4ac5-a53b-86857c3eec13 None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:16.842531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86266b98-b754-4ac5-a53b-86857c3eec13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:16.842854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86266b98-b754-4ac5-a53b-86857c3eec13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:16.844218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86266b98-b754-4ac5-a53b-86857c3eec13 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:16.852160 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 617/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:06:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:16.857866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495765af-c2b5-46f4-857d-afa832008c85 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:16.935094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:16.956838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495765af-c2b5-46f4-857d-afa832008c85 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:16.989721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b850c7-dcc8-4b71-8b75-08ee2b628006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:16.990533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b850c7-dcc8-4b71-8b75-08ee2b628006 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:16.990818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b850c7-dcc8-4b71-8b75-08ee2b628006 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:16.991101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b850c7-dcc8-4b71-8b75-08ee2b628006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:17.001391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b850c7-dcc8-4b71-8b75-08ee2b628006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:17.002896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b850c7-dcc8-4b71-8b75-08ee2b628006 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-64b850c7-dcc8-4b71-8b75-08ee2b628006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:17.020266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:17.021236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:17.022188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:17.022188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:17.022188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:17.022188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:17.022188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 634/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 12:06:17.028874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266b98-b754-4ac5-a53b-86857c3eec13 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:17.127916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266b98-b754-4ac5-a53b-86857c3eec13 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:17.161372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f56a2-70a8-4d43-b822-924036b11691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:17.162037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f56a2-70a8-4d43-b822-924036b11691 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:17.162319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f56a2-70a8-4d43-b822-924036b11691 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:17.162628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f56a2-70a8-4d43-b822-924036b11691 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:17.163343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f56a2-70a8-4d43-b822-924036b11691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:17.163654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f56a2-70a8-4d43-b822-924036b11691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:17.165015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f56a2-70a8-4d43-b822-924036b11691 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:17.177130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 618/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:17.183405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b850c7-dcc8-4b71-8b75-08ee2b628006 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:17.284333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b850c7-dcc8-4b71-8b75-08ee2b628006 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:17.309616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:17.317190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4033195-5a52-460e-b76f-7ba1dfacf895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:17.317984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4033195-5a52-460e-b76f-7ba1dfacf895 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:17.318268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4033195-5a52-460e-b76f-7ba1dfacf895 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:17.318574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4033195-5a52-460e-b76f-7ba1dfacf895 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:17.326474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4033195-5a52-460e-b76f-7ba1dfacf895 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:17.327915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4033195-5a52-460e-b76f-7ba1dfacf895 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a4033195-5a52-460e-b76f-7ba1dfacf895 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:06:17.342844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:17.344126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:17.345265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 635/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:17.349859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f56a2-70a8-4d43-b822-924036b11691 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:17.407561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:17.453074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f56a2-70a8-4d43-b822-924036b11691 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:17.486791 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8ab43-dd42-4aba-9980-f146b426b9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:17.487450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8ab43-dd42-4aba-9980-f146b426b9ab None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:17.487733 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8ab43-dd42-4aba-9980-f146b426b9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:17.488010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8ab43-dd42-4aba-9980-f146b426b9ab None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:17.488833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8ab43-dd42-4aba-9980-f146b426b9ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:17.489143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8ab43-dd42-4aba-9980-f146b426b9ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:17.490584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8ab43-dd42-4aba-9980-f146b426b9ab None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:17.507149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 619/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 12:06:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:17.513669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4033195-5a52-460e-b76f-7ba1dfacf895 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:17.616644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4033195-5a52-460e-b76f-7ba1dfacf895 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:17.649591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8da93-cb6e-4e2d-84c7-ad08223a3c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:17.650222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8da93-cb6e-4e2d-84c7-ad08223a3c14 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:17.651089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8da93-cb6e-4e2d-84c7-ad08223a3c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:17.651473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8da93-cb6e-4e2d-84c7-ad08223a3c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:17.709181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:17.726588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8da93-cb6e-4e2d-84c7-ad08223a3c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:17.728076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8da93-cb6e-4e2d-84c7-ad08223a3c14 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:17.826984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 636/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:06:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:06:17.832799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8ab43-dd42-4aba-9980-f146b426b9ab None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:17.934057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8ab43-dd42-4aba-9980-f146b426b9ab None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:17.946896 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:17.968219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a1645-c0c0-44bf-abf6-ce683471bbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:17.969005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a1645-c0c0-44bf-abf6-ce683471bbfc None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:17.969289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a1645-c0c0-44bf-abf6-ce683471bbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:17.969571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a1645-c0c0-44bf-abf6-ce683471bbfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:17.979326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a1645-c0c0-44bf-abf6-ce683471bbfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:17.980753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a1645-c0c0-44bf-abf6-ce683471bbfc None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-5f1a1645-c0c0-44bf-abf6-ce683471bbfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:17.997610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:17.998704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:17.999818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:17.999818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:17.999818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:17.999818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:17.999818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 620/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:18.004562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8da93-cb6e-4e2d-84c7-ad08223a3c14 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:18.092927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:18.104439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8da93-cb6e-4e2d-84c7-ad08223a3c14 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:18.138310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c6045-507e-40b5-9b5f-ad292d23f415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:18.139003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c6045-507e-40b5-9b5f-ad292d23f415 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:18.139293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c6045-507e-40b5-9b5f-ad292d23f415 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:18.139574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c6045-507e-40b5-9b5f-ad292d23f415 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:18.140297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c6045-507e-40b5-9b5f-ad292d23f415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:18.140610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c6045-507e-40b5-9b5f-ad292d23f415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:18.142038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c6045-507e-40b5-9b5f-ad292d23f415 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:18.152948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 637/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:18.158766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a1645-c0c0-44bf-abf6-ce683471bbfc None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:18.257213 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a1645-c0c0-44bf-abf6-ce683471bbfc None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:18.290844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ac440-2dee-4861-b4fa-7626536f5278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:18.291734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac440-2dee-4861-b4fa-7626536f5278 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:18.292012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac440-2dee-4861-b4fa-7626536f5278 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:18.292286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ac440-2dee-4861-b4fa-7626536f5278 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:18.299095 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ac440-2dee-4861-b4fa-7626536f5278 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:18.300532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ac440-2dee-4861-b4fa-7626536f5278 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-f86ac440-2dee-4861-b4fa-7626536f5278 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:06:18.314370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:18.315415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:18.316336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:06:18.316336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:06:18.316336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:06:18.316336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:06:18.316336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:06:18.316336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:06:18.316336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:18.316336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:18.316336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 621/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:06:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:18.321342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c6045-507e-40b5-9b5f-ad292d23f415 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:18.420248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c6045-507e-40b5-9b5f-ad292d23f415 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:18.453696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b816e2f-4372-4982-892c-d4ef1bf8a749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:18.454359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b816e2f-4372-4982-892c-d4ef1bf8a749 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:18.454691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b816e2f-4372-4982-892c-d4ef1bf8a749 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:18.454974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b816e2f-4372-4982-892c-d4ef1bf8a749 None admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:18.455799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b816e2f-4372-4982-892c-d4ef1bf8a749 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:18.456116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b816e2f-4372-4982-892c-d4ef1bf8a749 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:18.457555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b816e2f-4372-4982-892c-d4ef1bf8a749 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:18.469541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 638/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:06:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:18.475556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ac440-2dee-4861-b4fa-7626536f5278 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:18.502865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:18.579908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ac440-2dee-4861-b4fa-7626536f5278 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:18.615515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45004f-2e86-4f02-a584-0a7847ba6a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:18.616163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45004f-2e86-4f02-a584-0a7847ba6a22 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:18.616452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45004f-2e86-4f02-a584-0a7847ba6a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:18.616726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45004f-2e86-4f02-a584-0a7847ba6a22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:18.617554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45004f-2e86-4f02-a584-0a7847ba6a22 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=4f41920bad3147bba0000de911dfe5a1) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:18.617866 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45004f-2e86-4f02-a584-0a7847ba6a22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:18.619957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45004f-2e86-4f02-a584-0a7847ba6a22 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:18.625638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 622/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:06:18 2022] GET /identity/v3/role_assignments?user.id=4f41920bad3147bba0000de911dfe5a1&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=bd295cb5d1194b469beff0381c6b23af => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:20.818156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbf72c-c8bb-4598-b20f-fecb843b0548 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:20.818814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbf72c-c8bb-4598-b20f-fecb843b0548 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:20.818908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbf72c-c8bb-4598-b20f-fecb843b0548 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:20.820312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 639/1261] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:20.826196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196d2c-338d-4d5b-8081-ce639adc0129 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:20.826614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196d2c-338d-4d5b-8081-ce639adc0129 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:20.826795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196d2c-338d-4d5b-8081-ce639adc0129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:20.843927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1d196d2c-338d-4d5b-8081-ce639adc0129 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:20.845118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1d196d2c-338d-4d5b-8081-ce639adc0129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:20.910561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 623/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:20.934384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbf72c-c8bb-4598-b20f-fecb843b0548 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:20.945981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:21.034922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecbf72c-c8bb-4598-b20f-fecb843b0548 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:21.070272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de855e-9eaf-422a-b95e-ff3f0a4b7798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:21.071023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de855e-9eaf-422a-b95e-ff3f0a4b7798 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:21.071306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de855e-9eaf-422a-b95e-ff3f0a4b7798 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:21.071582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de855e-9eaf-422a-b95e-ff3f0a4b7798 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:21.081381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de855e-9eaf-422a-b95e-ff3f0a4b7798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-54de855e-9eaf-422a-b95e-ff3f0a4b7798 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:06:21.088978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:21.090067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:21.091230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 640/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:21.096117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d196d2c-338d-4d5b-8081-ce639adc0129 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:21.114883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:21.194567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d196d2c-338d-4d5b-8081-ce639adc0129 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:21.230522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d144c-d204-4a38-b749-df562624645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:21.231192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d144c-d204-4a38-b749-df562624645a None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:21.231478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d144c-d204-4a38-b749-df562624645a None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:21.231757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d144c-d204-4a38-b749-df562624645a None admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:21.232602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d144c-d204-4a38-b749-df562624645a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:21.232908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d144c-d204-4a38-b749-df562624645a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:21.234273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d144c-d204-4a38-b749-df562624645a None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:21.242169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 624/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:06:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:21.248225 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de855e-9eaf-422a-b95e-ff3f0a4b7798 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:21.315886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:21.348311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de855e-9eaf-422a-b95e-ff3f0a4b7798 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:21.380954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22235a1-a934-4fba-940c-ea977eba1f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:21.381736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22235a1-a934-4fba-940c-ea977eba1f35 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:21.382021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22235a1-a934-4fba-940c-ea977eba1f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:21.382299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22235a1-a934-4fba-940c-ea977eba1f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:21.392503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22235a1-a934-4fba-940c-ea977eba1f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:21.393938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22235a1-a934-4fba-940c-ea977eba1f35 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-c22235a1-a934-4fba-940c-ea977eba1f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:21.410940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:21.412084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:21.413241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:21.413241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:21.413241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:21.413241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:21.413241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 641/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:21.418734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d144c-d204-4a38-b749-df562624645a None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:21.500235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:21.519246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d144c-d204-4a38-b749-df562624645a None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:21.552867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a350e61-6ce5-4aeb-b279-cefe6f619c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:21.553525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a350e61-6ce5-4aeb-b279-cefe6f619c40 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:21.553809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a350e61-6ce5-4aeb-b279-cefe6f619c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:21.554083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a350e61-6ce5-4aeb-b279-cefe6f619c40 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:21.554873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a350e61-6ce5-4aeb-b279-cefe6f619c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:21.555180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a350e61-6ce5-4aeb-b279-cefe6f619c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:21.556531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a350e61-6ce5-4aeb-b279-cefe6f619c40 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:21.568623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 625/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:21.575095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22235a1-a934-4fba-940c-ea977eba1f35 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:21.676486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22235a1-a934-4fba-940c-ea977eba1f35 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:21.695825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:21.709984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066d22a-a79c-418a-894b-76575da74f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:21.710839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066d22a-a79c-418a-894b-76575da74f66 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:21.711123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066d22a-a79c-418a-894b-76575da74f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:21.711400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066d22a-a79c-418a-894b-76575da74f66 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:21.720485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066d22a-a79c-418a-894b-76575da74f66 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:21.721929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066d22a-a79c-418a-894b-76575da74f66 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-3066d22a-a79c-418a-894b-76575da74f66 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:06:21.736101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:21.737205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:21.738238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 642/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:21.745771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a350e61-6ce5-4aeb-b279-cefe6f619c40 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:21.843964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a350e61-6ce5-4aeb-b279-cefe6f619c40 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:21.878468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b82fb-1157-4b14-a310-f4c2c3b03fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:21.879131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b82fb-1157-4b14-a310-f4c2c3b03fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:21.879419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b82fb-1157-4b14-a310-f4c2c3b03fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:21.879692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b82fb-1157-4b14-a310-f4c2c3b03fe2 None admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:21.880518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b82fb-1157-4b14-a310-f4c2c3b03fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:21.880825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b82fb-1157-4b14-a310-f4c2c3b03fe2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:21.882284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b82fb-1157-4b14-a310-f4c2c3b03fe2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:21.891456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:21.900044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 626/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 12:06:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:21.910840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066d22a-a79c-418a-894b-76575da74f66 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:22.006070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066d22a-a79c-418a-894b-76575da74f66 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:22.040168 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae9cb1-a372-4275-ba3e-c4d8417d7b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:22.040949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae9cb1-a372-4275-ba3e-c4d8417d7b78 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:22.041293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae9cb1-a372-4275-ba3e-c4d8417d7b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:22.041564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae9cb1-a372-4275-ba3e-c4d8417d7b78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:22.052080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae9cb1-a372-4275-ba3e-c4d8417d7b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:22.053506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae9cb1-a372-4275-ba3e-c4d8417d7b78 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-05ae9cb1-a372-4275-ba3e-c4d8417d7b78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:22.070576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:22.073547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:22.074584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:22.074584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:22.074584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:22.074584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:22.074584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 643/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:22.078526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b82fb-1157-4b14-a310-f4c2c3b03fe2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:22.183147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b82fb-1157-4b14-a310-f4c2c3b03fe2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:22.217327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2318d7-aa58-436d-83cc-acdf34aa51c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:22.217974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2318d7-aa58-436d-83cc-acdf34aa51c1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:22.218257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2318d7-aa58-436d-83cc-acdf34aa51c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:22.218560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2318d7-aa58-436d-83cc-acdf34aa51c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:22.219272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2318d7-aa58-436d-83cc-acdf34aa51c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:22.219578 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2318d7-aa58-436d-83cc-acdf34aa51c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:22.221032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2318d7-aa58-436d-83cc-acdf34aa51c1 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:22.232079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 627/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:22.237694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae9cb1-a372-4275-ba3e-c4d8417d7b78 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:22.270173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:22.338891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae9cb1-a372-4275-ba3e-c4d8417d7b78 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:22.373178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8646bf2-df80-4707-af91-60cbe0b927d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:22.373967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8646bf2-df80-4707-af91-60cbe0b927d0 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:22.374248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8646bf2-df80-4707-af91-60cbe0b927d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:22.374565 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8646bf2-df80-4707-af91-60cbe0b927d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:22.381850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8646bf2-df80-4707-af91-60cbe0b927d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:22.383690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8646bf2-df80-4707-af91-60cbe0b927d0 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-a8646bf2-df80-4707-af91-60cbe0b927d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:06:22.399542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:22.401319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:22.402399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:06:22.402399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:06:22.402399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:06:22.402399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:06:22.402399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:06:22.402399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:06:22.402399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:22.402399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:22.402399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 644/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:22.408178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2318d7-aa58-436d-83cc-acdf34aa51c1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:22.468059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:22.509263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2318d7-aa58-436d-83cc-acdf34aa51c1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:22.544622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e9844-8f28-47fb-890f-3570d2a706af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:22.545211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e9844-8f28-47fb-890f-3570d2a706af None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:22.545492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e9844-8f28-47fb-890f-3570d2a706af None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:22.545764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e9844-8f28-47fb-890f-3570d2a706af None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:22.546659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e9844-8f28-47fb-890f-3570d2a706af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:22.546971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e9844-8f28-47fb-890f-3570d2a706af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:22.548392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e9844-8f28-47fb-890f-3570d2a706af None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:22.561370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 628/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:06:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:06:22.567322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8646bf2-df80-4707-af91-60cbe0b927d0 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:22.658605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:22.667273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8646bf2-df80-4707-af91-60cbe0b927d0 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:22.700940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e03108-aee7-4f47-86c1-d7b41d7d1ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:22.701825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03108-aee7-4f47-86c1-d7b41d7d1ea1 None admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:22.702111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03108-aee7-4f47-86c1-d7b41d7d1ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:22.702394 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e03108-aee7-4f47-86c1-d7b41d7d1ea1 None admin] PATH_INFO: `/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/4f41920bad3147bba0000de911dfe5a1/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:22.704365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e03108-aee7-4f47-86c1-d7b41d7d1ea1 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ce53bfef52443580430ddb1eb6185a, user_id=4f41920bad3147bba0000de911dfe5a1, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:22.705794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e03108-aee7-4f47-86c1-d7b41d7d1ea1 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:22.796501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 645/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 19 12:06:22 2022] PUT /identity/v3/projects/94ce53bfef52443580430ddb1eb6185a/users/4f41920bad3147bba0000de911dfe5a1/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:06:24.995041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4d2c0-15dc-4e46-afb5-6240f34cac7f None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:24.995635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4d2c0-15dc-4e46-afb5-6240f34cac7f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:24.995635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4d2c0-15dc-4e46-afb5-6240f34cac7f None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:24.996954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 629/1274] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:25.002633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcd8ea-bcf4-4c20-bcc2-9432a3979029 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:25.003066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcd8ea-bcf4-4c20-bcc2-9432a3979029 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:25.003197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcd8ea-bcf4-4c20-bcc2-9432a3979029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:25.021020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-77fcd8ea-bcf4-4c20-bcc2-9432a3979029 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:25.022209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-77fcd8ea-bcf4-4c20-bcc2-9432a3979029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:25.088532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 646/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:25.112303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4d2c0-15dc-4e46-afb5-6240f34cac7f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:25.215981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4d2c0-15dc-4e46-afb5-6240f34cac7f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:25.234305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:25.251277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b050c-312b-49ea-80fb-0bfddb9381c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:25.252009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b050c-312b-49ea-80fb-0bfddb9381c1 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:25.252299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b050c-312b-49ea-80fb-0bfddb9381c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:25.252580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b050c-312b-49ea-80fb-0bfddb9381c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:25.263320 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b050c-312b-49ea-80fb-0bfddb9381c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-c53b050c-312b-49ea-80fb-0bfddb9381c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 12:06:25.271274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:25.282779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:25.283720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 630/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 19 12:06:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:25.284481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcd8ea-bcf4-4c20-bcc2-9432a3979029 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:25.360696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:25.378773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcd8ea-bcf4-4c20-bcc2-9432a3979029 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:25.411591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efd92a-5ffc-4deb-a33f-1a58ee6caac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:25.418688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd92a-5ffc-4deb-a33f-1a58ee6caac6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:25.418688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd92a-5ffc-4deb-a33f-1a58ee6caac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:25.418688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd92a-5ffc-4deb-a33f-1a58ee6caac6 None admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:25.418688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efd92a-5ffc-4deb-a33f-1a58ee6caac6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:25.418688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efd92a-5ffc-4deb-a33f-1a58ee6caac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:25.418688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efd92a-5ffc-4deb-a33f-1a58ee6caac6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:25.423527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 647/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 19 12:06:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:25.429189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b050c-312b-49ea-80fb-0bfddb9381c1 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:25.530146 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b050c-312b-49ea-80fb-0bfddb9381c1 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:25.563971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f743d-f840-4210-b55a-b6f31ac919a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:25.564768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f743d-f840-4210-b55a-b6f31ac919a2 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:25.565050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f743d-f840-4210-b55a-b6f31ac919a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:25.565323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f743d-f840-4210-b55a-b6f31ac919a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:25.576789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f743d-f840-4210-b55a-b6f31ac919a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:25.579146 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7f743d-f840-4210-b55a-b6f31ac919a2 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-fb7f743d-f840-4210-b55a-b6f31ac919a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:25.595325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:25.596396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:25.597353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:25.597353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:25.597353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:25.597353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:25.597353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 631/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:25.602319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efd92a-5ffc-4deb-a33f-1a58ee6caac6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:25.700669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efd92a-5ffc-4deb-a33f-1a58ee6caac6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:25.734394 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a601-f15c-4d6e-a8d9-ded75d4889f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:25.735073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a601-f15c-4d6e-a8d9-ded75d4889f6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:25.735355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a601-f15c-4d6e-a8d9-ded75d4889f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:25.735627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a601-f15c-4d6e-a8d9-ded75d4889f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:25.736336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861a601-f15c-4d6e-a8d9-ded75d4889f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:25.736644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861a601-f15c-4d6e-a8d9-ded75d4889f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:25.737996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861a601-f15c-4d6e-a8d9-ded75d4889f6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:25.749930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 648/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:25.755766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f743d-f840-4210-b55a-b6f31ac919a2 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:25.792231 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:25.855370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f743d-f840-4210-b55a-b6f31ac919a2 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:25.889476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edafac-bdac-4d5c-92f7-4efc34102054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:25.890268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edafac-bdac-4d5c-92f7-4efc34102054 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:25.890585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edafac-bdac-4d5c-92f7-4efc34102054 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:25.890866 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74edafac-bdac-4d5c-92f7-4efc34102054 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:25.898422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edafac-bdac-4d5c-92f7-4efc34102054 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:25.899841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74edafac-bdac-4d5c-92f7-4efc34102054 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-74edafac-bdac-4d5c-92f7-4efc34102054 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 12:06:25.915517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:25.917174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:25.918431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 632/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:25.923671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a601-f15c-4d6e-a8d9-ded75d4889f6 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:25.927455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:26.021924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a601-f15c-4d6e-a8d9-ded75d4889f6 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:26.059091 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c517ef-cdf7-4a08-b0f8-dc729c044cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:26.059678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c517ef-cdf7-4a08-b0f8-dc729c044cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:26.059962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c517ef-cdf7-4a08-b0f8-dc729c044cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:26.060242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c517ef-cdf7-4a08-b0f8-dc729c044cb7 None admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:26.061076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c517ef-cdf7-4a08-b0f8-dc729c044cb7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:26.061397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c517ef-cdf7-4a08-b0f8-dc729c044cb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:26.062853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c517ef-cdf7-4a08-b0f8-dc729c044cb7 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:26.082616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 649/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 19 12:06:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:26.090473 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edafac-bdac-4d5c-92f7-4efc34102054 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:26.183367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:26.192788 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74edafac-bdac-4d5c-92f7-4efc34102054 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:26.226161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254ecb8-d60c-4038-b5fb-e6c44fe0d6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:26.226848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254ecb8-d60c-4038-b5fb-e6c44fe0d6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:26.227137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254ecb8-d60c-4038-b5fb-e6c44fe0d6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:26.227417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254ecb8-d60c-4038-b5fb-e6c44fe0d6f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:26.295341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254ecb8-d60c-4038-b5fb-e6c44fe0d6f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:26.296790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254ecb8-d60c-4038-b5fb-e6c44fe0d6f3 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:26.384563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:26.395791 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 633/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 19 12:06:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 12:06:26.402461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c517ef-cdf7-4a08-b0f8-dc729c044cb7 None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:26.465979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:26.504586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c517ef-cdf7-4a08-b0f8-dc729c044cb7 None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:26.543627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaca75-7414-4735-acaa-85228363a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:26.544517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaca75-7414-4735-acaa-85228363a7bd None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:26.544797 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaca75-7414-4735-acaa-85228363a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:26.545072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaca75-7414-4735-acaa-85228363a7bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:26.555738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaca75-7414-4735-acaa-85228363a7bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:26.559257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaca75-7414-4735-acaa-85228363a7bd None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-79aaca75-7414-4735-acaa-85228363a7bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 12:06:26.575852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:26.576940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:26.578297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 12:06:26.578297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 12:06:26.578297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 12:06:26.578297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:26.578297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 650/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:26.582830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254ecb8-d60c-4038-b5fb-e6c44fe0d6f3 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:26.681292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254ecb8-d60c-4038-b5fb-e6c44fe0d6f3 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:26.715851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f991b3-9bdb-4247-b977-4d538e03922f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:26.716535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f991b3-9bdb-4247-b977-4d538e03922f None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:26.716822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f991b3-9bdb-4247-b977-4d538e03922f None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:26.717103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f991b3-9bdb-4247-b977-4d538e03922f None admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:26.718067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f991b3-9bdb-4247-b977-4d538e03922f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:26.718572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f991b3-9bdb-4247-b977-4d538e03922f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:26.720728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f991b3-9bdb-4247-b977-4d538e03922f None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:26.736973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 634/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 19 12:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:26.742635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaca75-7414-4735-acaa-85228363a7bd None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:26.846577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaca75-7414-4735-acaa-85228363a7bd None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:26.860510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:26.880551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2c737-37ee-4543-91bc-7aa1ffd6adba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:26.881337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2c737-37ee-4543-91bc-7aa1ffd6adba None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:26.881623 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2c737-37ee-4543-91bc-7aa1ffd6adba None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:26.881902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2c737-37ee-4543-91bc-7aa1ffd6adba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:26.889121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2c737-37ee-4543-91bc-7aa1ffd6adba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:26.890604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2c737-37ee-4543-91bc-7aa1ffd6adba None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application [None req-e0c2c737-37ee-4543-91bc-7aa1ffd6adba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 12:06:26.904371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application with Lock( May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:26.905290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:26.906334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 12:06:26.906334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 12:06:26.906334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 12:06:26.906334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 12:06:26.906334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 12:06:26.906334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 12:06:26.906334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 12:06:26.906334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: ERROR keystone.server.flask.application May 19 12:06:26.906334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 651/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 19 12:06:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:26.911356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f991b3-9bdb-4247-b977-4d538e03922f None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:26.961571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:27.008684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f991b3-9bdb-4247-b977-4d538e03922f None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:27.042688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d667950-41c3-4d5b-9db5-a329872a962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:27.043611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d667950-41c3-4d5b-9db5-a329872a962d None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:27.043891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d667950-41c3-4d5b-9db5-a329872a962d None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:27.044166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d667950-41c3-4d5b-9db5-a329872a962d None admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:27.044937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d667950-41c3-4d5b-9db5-a329872a962d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:27.045248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d667950-41c3-4d5b-9db5-a329872a962d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:27.046692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d667950-41c3-4d5b-9db5-a329872a962d None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:27.059018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 635/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 19 12:06:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:27.064908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2c737-37ee-4543-91bc-7aa1ffd6adba None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:27.162365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2c737-37ee-4543-91bc-7aa1ffd6adba None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:27.196385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c9fb4-2582-4829-861c-4e34f691d8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:27.205379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c9fb4-2582-4829-861c-4e34f691d8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:27.205379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c9fb4-2582-4829-861c-4e34f691d8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:27.205379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c9fb4-2582-4829-861c-4e34f691d8e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:27.205379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c9fb4-2582-4829-861c-4e34f691d8e5 None admin] RBAC: Adding query filter params (role.id=bd295cb5d1194b469beff0381c6b23af, scope.project.id=94ce53bfef52443580430ddb1eb6185a, user.id=4f41920bad3147bba0000de911dfe5a1) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:27.205379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c9fb4-2582-4829-861c-4e34f691d8e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:27.205379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c9fb4-2582-4829-861c-4e34f691d8e5 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:27.206233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 652/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 19 12:06:27 2022] GET /identity/v3/role_assignments?user.id=4f41920bad3147bba0000de911dfe5a1&scope.project.id=94ce53bfef52443580430ddb1eb6185a&role.id=bd295cb5d1194b469beff0381c6b23af => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:29.456172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482912f0-0023-4a58-957a-7c87838d4794 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:29.456699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482912f0-0023-4a58-957a-7c87838d4794 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:29.456699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482912f0-0023-4a58-957a-7c87838d4794 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:29.458122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 636/1288] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:06:29.464017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9b8fa-2bf8-4871-8f21-f635c2d159ef None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:29.466638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9b8fa-2bf8-4871-8f21-f635c2d159ef None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:29.466638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9b8fa-2bf8-4871-8f21-f635c2d159ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:29.480983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c3a9b8fa-2bf8-4871-8f21-f635c2d159ef None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:29.482159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c3a9b8fa-2bf8-4871-8f21-f635c2d159ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:29.511514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:29.549522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 653/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:29.571710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482912f0-0023-4a58-957a-7c87838d4794 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:29.678396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482912f0-0023-4a58-957a-7c87838d4794 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:29.689469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:29.713827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7961149-a84e-4443-9b3b-b0127b7ac6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:29.714575 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7961149-a84e-4443-9b3b-b0127b7ac6db None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:29.714863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7961149-a84e-4443-9b3b-b0127b7ac6db None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:29.715144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7961149-a84e-4443-9b3b-b0127b7ac6db None admin] PATH_INFO: `/v3/services/placement` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:29.715794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7961149-a84e-4443-9b3b-b0127b7ac6db None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:29.717163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7961149-a84e-4443-9b3b-b0127b7ac6db None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-d7961149-a84e-4443-9b3b-b0127b7ac6db None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:29.731112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:29.732433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:29.733286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 637/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:06:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:29.738648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9b8fa-2bf8-4871-8f21-f635c2d159ef None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:29.847220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9b8fa-2bf8-4871-8f21-f635c2d159ef None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:29.880688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59545a82-db91-4037-a7b7-164cbd938209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:29.881345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545a82-db91-4037-a7b7-164cbd938209 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:29.881631 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545a82-db91-4037-a7b7-164cbd938209 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:29.881906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545a82-db91-4037-a7b7-164cbd938209 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:29.882632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59545a82-db91-4037-a7b7-164cbd938209 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:29.882943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59545a82-db91-4037-a7b7-164cbd938209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:29.884321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59545a82-db91-4037-a7b7-164cbd938209 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:29.889835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 654/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 12:06:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:29.895584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7961149-a84e-4443-9b3b-b0127b7ac6db None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:29.997211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7961149-a84e-4443-9b3b-b0127b7ac6db None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:30.030659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3b40a-fbe0-4fb3-ab4d-c87dd603b6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:30.031322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3b40a-fbe0-4fb3-ab4d-c87dd603b6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:30.031607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3b40a-fbe0-4fb3-ab4d-c87dd603b6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:30.032029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3b40a-fbe0-4fb3-ab4d-c87dd603b6c5 None admin] PATH_INFO: `/v3/services` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:30.032652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3b40a-fbe0-4fb3-ab4d-c87dd603b6c5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:30.032962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3b40a-fbe0-4fb3-ab4d-c87dd603b6c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:30.034318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3b40a-fbe0-4fb3-ab4d-c87dd603b6c5 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:30.038864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 638/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 12:06:29 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:32.316960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e847e-6180-4356-9b63-1522e456d287 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:32.317469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e847e-6180-4356-9b63-1522e456d287 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:32.317469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e847e-6180-4356-9b63-1522e456d287 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:32.318885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 655/1293] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:32.325032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768886d-1b7d-473f-b844-a904f3f376fb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:32.325433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768886d-1b7d-473f-b844-a904f3f376fb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:32.325624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768886d-1b7d-473f-b844-a904f3f376fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:32.342437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7768886d-1b7d-473f-b844-a904f3f376fb None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:32.343787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7768886d-1b7d-473f-b844-a904f3f376fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:32.373694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:32.410423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 639/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:32.434167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e847e-6180-4356-9b63-1522e456d287 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:32.464934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:32.533547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e847e-6180-4356-9b63-1522e456d287 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:32.568442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1b8e7-2d0f-4d00-b311-52b96812f13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:32.569016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1b8e7-2d0f-4d00-b311-52b96812f13d None admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:32.569299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1b8e7-2d0f-4d00-b311-52b96812f13d None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:32.569570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1b8e7-2d0f-4d00-b311-52b96812f13d None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:32.570286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1b8e7-2d0f-4d00-b311-52b96812f13d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:32.571746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde1b8e7-2d0f-4d00-b311-52b96812f13d None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:32.693967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 656/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 19 12:06:32 2022] POST /identity/v3/services => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:06:34.919224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a2536-7bf7-415c-bc23-c882352c3c18 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:34.919784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a2536-7bf7-415c-bc23-c882352c3c18 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:34.919784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a2536-7bf7-415c-bc23-c882352c3c18 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:34.921143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 640/1296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:34.926725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42a60a-19f8-4d93-8062-0a6d7a59cdba None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:34.927173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42a60a-19f8-4d93-8062-0a6d7a59cdba None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:34.927279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42a60a-19f8-4d93-8062-0a6d7a59cdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:34.944848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2d42a60a-19f8-4d93-8062-0a6d7a59cdba None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:34.946025 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2d42a60a-19f8-4d93-8062-0a6d7a59cdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:35.016878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 657/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:34 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:35.039229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a2536-7bf7-415c-bc23-c882352c3c18 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:35.141800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a2536-7bf7-415c-bc23-c882352c3c18 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:35.176038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fa06f-f077-4c8c-8a28-49f9362cf553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:35.176764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fa06f-f077-4c8c-8a28-49f9362cf553 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:35.177044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fa06f-f077-4c8c-8a28-49f9362cf553 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:35.177315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fa06f-f077-4c8c-8a28-49f9362cf553 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:35.177957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fa06f-f077-4c8c-8a28-49f9362cf553 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:35.179333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fa06f-f077-4c8c-8a28-49f9362cf553 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-941fa06f-f077-4c8c-8a28-49f9362cf553 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:35.191356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:35.198336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:35.199465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 641/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:06:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:35.201523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42a60a-19f8-4d93-8062-0a6d7a59cdba None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:35.220703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:35.298743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42a60a-19f8-4d93-8062-0a6d7a59cdba None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:35.331511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45553a05-8f2e-4d3c-8d9b-63f219f272de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:35.331951 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45553a05-8f2e-4d3c-8d9b-63f219f272de None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:35.332163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45553a05-8f2e-4d3c-8d9b-63f219f272de None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:35.332442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45553a05-8f2e-4d3c-8d9b-63f219f272de None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:35.333143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45553a05-8f2e-4d3c-8d9b-63f219f272de None admin] RBAC: Adding query filter params (name=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:35.333451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45553a05-8f2e-4d3c-8d9b-63f219f272de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:35.334861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45553a05-8f2e-4d3c-8d9b-63f219f272de None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:35.340451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 658/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 12:06:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:35.346400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fa06f-f077-4c8c-8a28-49f9362cf553 None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:35.379166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:35.446933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fa06f-f077-4c8c-8a28-49f9362cf553 None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:35.479154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9e48a-1ebf-4221-9ad7-7234212720b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:35.479815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9e48a-1ebf-4221-9ad7-7234212720b6 None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:35.483113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9e48a-1ebf-4221-9ad7-7234212720b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:35.483113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9e48a-1ebf-4221-9ad7-7234212720b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:35.483113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9e48a-1ebf-4221-9ad7-7234212720b6 None admin] RBAC: Adding query filter params (interface=public, service_id=a74df924bf064eb8a0c2f78c2cc2ac52, region_id=RegionOne) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:35.483113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9e48a-1ebf-4221-9ad7-7234212720b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:35.483113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9e48a-1ebf-4221-9ad7-7234212720b6 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:35.487791 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 642/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 19 12:06:35 2022] GET /identity/v3/endpoints?service_id=a74df924bf064eb8a0c2f78c2cc2ac52&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:35.493457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45553a05-8f2e-4d3c-8d9b-63f219f272de None admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:35.594197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45553a05-8f2e-4d3c-8d9b-63f219f272de None admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:35.627529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f28bf9-05f2-4b52-a1e5-f30bea3826a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:35.628185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28bf9-05f2-4b52-a1e5-f30bea3826a2 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:35.628470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28bf9-05f2-4b52-a1e5-f30bea3826a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:35.628745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28bf9-05f2-4b52-a1e5-f30bea3826a2 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:35.629415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f28bf9-05f2-4b52-a1e5-f30bea3826a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:35.630815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f28bf9-05f2-4b52-a1e5-f30bea3826a2 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:35.637574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 659/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 19 12:06:35 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:06:35.697692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:37.887986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877b599-71fc-44b3-91e6-34644a70ddee None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:37.888579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877b599-71fc-44b3-91e6-34644a70ddee None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:37.888579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e877b599-71fc-44b3-91e6-34644a70ddee None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:37.889900 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 643/1302] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:06:37.895410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89334865-cdb2-4ed6-965b-2bd566b4412a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:37.895949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89334865-cdb2-4ed6-965b-2bd566b4412a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:37.895949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89334865-cdb2-4ed6-965b-2bd566b4412a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:37.914052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-89334865-cdb2-4ed6-965b-2bd566b4412a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:06:37.915376 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-89334865-cdb2-4ed6-965b-2bd566b4412a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:06:37.984650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 660/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:06:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 12:06:38.007352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877b599-71fc-44b3-91e6-34644a70ddee None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:38.114858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:38.122206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e877b599-71fc-44b3-91e6-34644a70ddee None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:38.158920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431365c-46af-41d3-9160-d4a2e04308da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:38.159645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431365c-46af-41d3-9160-d4a2e04308da None admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:38.159931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431365c-46af-41d3-9160-d4a2e04308da None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:38.160203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431365c-46af-41d3-9160-d4a2e04308da None admin] PATH_INFO: `/v3/services/placement` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:38.160876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431365c-46af-41d3-9160-d4a2e04308da None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:38.162250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431365c-46af-41d3-9160-d4a2e04308da None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application [None req-f431365c-46af-41d3-9160-d4a2e04308da None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 12:06:38.175780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 12:06:38.177098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.get_or_create( May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application with Lock( May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self._enter() May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application return self.creator() May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application created_value = creator( May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: ERROR keystone.server.flask.application May 19 12:06:38.177982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 644/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 19 12:06:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 12:06:38.183936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89334865-cdb2-4ed6-965b-2bd566b4412a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:38.264868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:38.290396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89334865-cdb2-4ed6-965b-2bd566b4412a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:38.332441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a5ec6-b27c-4d36-9c14-96ad3ff69de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:38.333129 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a5ec6-b27c-4d36-9c14-96ad3ff69de6 None admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:38.333413 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a5ec6-b27c-4d36-9c14-96ad3ff69de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:38.333687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a5ec6-b27c-4d36-9c14-96ad3ff69de6 None admin] PATH_INFO: `/v3/services` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:38.334432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a5ec6-b27c-4d36-9c14-96ad3ff69de6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:06:38.334743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a5ec6-b27c-4d36-9c14-96ad3ff69de6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:38.336145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a5ec6-b27c-4d36-9c14-96ad3ff69de6 None admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:38.342871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 661/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 19 12:06:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:06:38.348684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431365c-46af-41d3-9160-d4a2e04308da None admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:06:38.456556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431365c-46af-41d3-9160-d4a2e04308da None admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:06:38.491758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e892f8-f5da-437c-9b0d-9c28a974a605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:06:38.501137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e892f8-f5da-437c-9b0d-9c28a974a605 None admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:06:38.501137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e892f8-f5da-437c-9b0d-9c28a974a605 None admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:06:38.501137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e892f8-f5da-437c-9b0d-9c28a974a605 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:06:38.501137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e892f8-f5da-437c-9b0d-9c28a974a605 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:06:38.501137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e892f8-f5da-437c-9b0d-9c28a974a605 None admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:06:38.665265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 645/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 19 12:06:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:06:48.481629 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=30 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:06:48.660398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:51.353531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62d7a1-d95b-4084-bd44-49c3c0cef2ca None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:51.354027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62d7a1-d95b-4084-bd44-49c3c0cef2ca None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:51.354119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62d7a1-d95b-4084-bd44-49c3c0cef2ca None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:51.355755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 662/1307] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:14:51.362348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb0b05-c9c3-4d04-93d1-8f0a7a91450a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:51.397568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb0b05-c9c3-4d04-93d1-8f0a7a91450a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:51.397568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb0b05-c9c3-4d04-93d1-8f0a7a91450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:51.456775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9ceb0b05-c9c3-4d04-93d1-8f0a7a91450a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:14:51.458010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9ceb0b05-c9c3-4d04-93d1-8f0a7a91450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:51.587467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 646/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:14:51.624114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6f98e-3020-4288-879d-f6b899e7ea19 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:51.624691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6f98e-3020-4288-879d-f6b899e7ea19 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:51.625125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d6f98e-3020-4288-879d-f6b899e7ea19 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:51.627417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 663/1309] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 12:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:14:51.633884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d32b60-2aa9-4279-ba9a-746d0e569747 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:51.643397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d32b60-2aa9-4279-ba9a-746d0e569747 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:51.643397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d32b60-2aa9-4279-ba9a-746d0e569747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:51.726043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-23d32b60-2aa9-4279-ba9a-746d0e569747 None None] MFA Rules not processed for user `8b266ea9964c4a179c2743b9cf6478eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:14:51.727310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-23d32b60-2aa9-4279-ba9a-746d0e569747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:51.749661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:51.812682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 647/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 12:14:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:14:51.822078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6f98e-3020-4288-879d-f6b899e7ea19 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:14:51.823844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-98d6f98e-3020-4288-879d-f6b899e7ea19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:51.941217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d6f98e-3020-4288-879d-f6b899e7ea19 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:14:52.020162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee2757-f894-4f15-b9e0-97bab4afc1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:14:52.020837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee2757-f894-4f15-b9e0-97bab4afc1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:52.021120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee2757-f894-4f15-b9e0-97bab4afc1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:52.021399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee2757-f894-4f15-b9e0-97bab4afc1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:52.023089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-83ee2757-f894-4f15-b9e0-97bab4afc1c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:52.082787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:52.143875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee2757-f894-4f15-b9e0-97bab4afc1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:14:52.145317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee2757-f894-4f15-b9e0-97bab4afc1c4 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:14:52.165741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 664/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:14:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:14:54.890169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be971ad5-59d9-41d7-8093-34abce50566a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:54.890646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be971ad5-59d9-41d7-8093-34abce50566a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:54.890646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be971ad5-59d9-41d7-8093-34abce50566a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:54.892729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 648/1312] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:14:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:14:54.898325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ab83-cff5-4552-a29f-f605bc2b2171 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:54.898905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ab83-cff5-4552-a29f-f605bc2b2171 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:54.899068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ab83-cff5-4552-a29f-f605bc2b2171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:54.976326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3c27ab83-cff5-4552-a29f-f605bc2b2171 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:14:54.977570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3c27ab83-cff5-4552-a29f-f605bc2b2171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:55.083892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 665/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:14:55.136894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e181a-51ec-4b2f-9699-b01bf6883e6c None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:55.137185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e181a-51ec-4b2f-9699-b01bf6883e6c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:55.137466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e181a-51ec-4b2f-9699-b01bf6883e6c None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:55.138890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 649/1314] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 19 12:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:14:55.145612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625ecd-5947-4de7-8aec-78a1d7a46c44 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:55.145909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625ecd-5947-4de7-8aec-78a1d7a46c44 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:55.146192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625ecd-5947-4de7-8aec-78a1d7a46c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:55.157931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:55.243284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3b625ecd-5947-4de7-8aec-78a1d7a46c44 None None] MFA Rules not processed for user `8b266ea9964c4a179c2743b9cf6478eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:14:55.244527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3b625ecd-5947-4de7-8aec-78a1d7a46c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:55.332117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 666/1315] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 19 12:14:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:14:55.341593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e181a-51ec-4b2f-9699-b01bf6883e6c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:14:55.343286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-008e181a-51ec-4b2f-9699-b01bf6883e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:55.459822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e181a-51ec-4b2f-9699-b01bf6883e6c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:14:55.537755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:55.540583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f61c2-3c19-41e4-af29-a5032b1c234c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:14:55.541531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f61c2-3c19-41e4-af29-a5032b1c234c service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:55.541886 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f61c2-3c19-41e4-af29-a5032b1c234c service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:55.542229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f61c2-3c19-41e4-af29-a5032b1c234c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:55.544021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-344f61c2-3c19-41e4-af29-a5032b1c234c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:55.674242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f61c2-3c19-41e4-af29-a5032b1c234c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:14:55.675874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f61c2-3c19-41e4-af29-a5032b1c234c service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:14:55.698258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 650/1316] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:14:58.798995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc222d-c9b3-4e54-a420-b9ba54397f24 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:58.798995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc222d-c9b3-4e54-a420-b9ba54397f24 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:58.798995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc222d-c9b3-4e54-a420-b9ba54397f24 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:58.798995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 667/1317] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:14:58.803331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1356221-e073-44f3-9b3d-db5fdf3f501d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:58.803756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1356221-e073-44f3-9b3d-db5fdf3f501d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:58.803897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1356221-e073-44f3-9b3d-db5fdf3f501d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:58.874965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e1356221-e073-44f3-9b3d-db5fdf3f501d None None] MFA Rules not processed for user `d695ee7d689a4ca4ae7d161f5773ca60`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:14:58.876909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e1356221-e073-44f3-9b3d-db5fdf3f501d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:58.931517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:58.982145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 651/1318] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:14:59.024877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc222d-c9b3-4e54-a420-b9ba54397f24 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:14:59.026452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-76fc222d-c9b3-4e54-a420-b9ba54397f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:59.132429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc222d-c9b3-4e54-a420-b9ba54397f24 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:14:59.135306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:14:59.209149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d69fb3-3fb8-4016-a19f-004a25877b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:14:59.209801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d69fb3-3fb8-4016-a19f-004a25877b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:14:59.210093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d69fb3-3fb8-4016-a19f-004a25877b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:14:59.210439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d69fb3-3fb8-4016-a19f-004a25877b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:14:59.212118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b1d69fb3-3fb8-4016-a19f-004a25877b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:14:59.326196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d69fb3-3fb8-4016-a19f-004a25877b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:14:59.327680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d69fb3-3fb8-4016-a19f-004a25877b85 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:14:59.346347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 668/1319] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:14:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:02.618881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb9a96-2326-4d91-8f14-c596c9f1a32f None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:02.619422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb9a96-2326-4d91-8f14-c596c9f1a32f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:02.619422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb9a96-2326-4d91-8f14-c596c9f1a32f None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:02.620781 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 652/1320] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:15:02.627112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3a83c-4bfe-408f-a7b5-deb279e80997 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:02.627560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3a83c-4bfe-408f-a7b5-deb279e80997 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:02.627784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3a83c-4bfe-408f-a7b5-deb279e80997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:02.667670 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:02.694228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4da3a83c-4bfe-408f-a7b5-deb279e80997 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:02.695723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4da3a83c-4bfe-408f-a7b5-deb279e80997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:02.792383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 669/1321] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:02.831439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb9a96-2326-4d91-8f14-c596c9f1a32f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:02.832977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-97fb9a96-2326-4d91-8f14-c596c9f1a32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:02.936909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb9a96-2326-4d91-8f14-c596c9f1a32f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:03.009794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c110ab-270e-44f0-8b39-d63533a83011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:03.010458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c110ab-270e-44f0-8b39-d63533a83011 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:03.010744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c110ab-270e-44f0-8b39-d63533a83011 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:03.011019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c110ab-270e-44f0-8b39-d63533a83011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:03.012697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d0c110ab-270e-44f0-8b39-d63533a83011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:03.019985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:03.126196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c110ab-270e-44f0-8b39-d63533a83011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:03.127746 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c110ab-270e-44f0-8b39-d63533a83011 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:03.146022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 653/1322] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:06.598064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eca152-817c-419f-873b-d7e73eb6643a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:06.598710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eca152-817c-419f-873b-d7e73eb6643a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:06.598710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eca152-817c-419f-873b-d7e73eb6643a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:06.600001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 670/1323] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:06.605625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b717fc-ea5b-4e5e-9b2f-0eb53620d625 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:06.606125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b717fc-ea5b-4e5e-9b2f-0eb53620d625 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:06.606300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b717fc-ea5b-4e5e-9b2f-0eb53620d625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:06.680764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e5b717fc-ea5b-4e5e-9b2f-0eb53620d625 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:06.681518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e5b717fc-ea5b-4e5e-9b2f-0eb53620d625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:06.744236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:06.789559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 654/1324] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:06.826217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eca152-817c-419f-873b-d7e73eb6643a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:06.827876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-92eca152-817c-419f-873b-d7e73eb6643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:06.935460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eca152-817c-419f-873b-d7e73eb6643a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:06.947939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:07.013501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c0b03-a9b1-45fd-8f81-ef6bc9050ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:07.014153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0b03-a9b1-45fd-8f81-ef6bc9050ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:07.014467 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0b03-a9b1-45fd-8f81-ef6bc9050ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:07.014753 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0b03-a9b1-45fd-8f81-ef6bc9050ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:07.016746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6a8c0b03-a9b1-45fd-8f81-ef6bc9050ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:07.131374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c0b03-a9b1-45fd-8f81-ef6bc9050ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:07.132822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c0b03-a9b1-45fd-8f81-ef6bc9050ce7 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:07.151664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 671/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:09.514037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb9182-7591-4534-88d7-dc1bb0466344 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:09.514627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb9182-7591-4534-88d7-dc1bb0466344 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:09.514627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb9182-7591-4534-88d7-dc1bb0466344 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:09.516153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 655/1326] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:15:09.522302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75a06-edbf-4d27-bcd0-b1e0083c43f0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:09.522912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75a06-edbf-4d27-bcd0-b1e0083c43f0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:09.522912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec75a06-edbf-4d27-bcd0-b1e0083c43f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:09.574191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:09.592519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-eec75a06-edbf-4d27-bcd0-b1e0083c43f0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:09.595420 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-eec75a06-edbf-4d27-bcd0-b1e0083c43f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:09.697257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 672/1327] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:09.733229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb9182-7591-4534-88d7-dc1bb0466344 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:09.735791 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-01eb9182-7591-4534-88d7-dc1bb0466344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:09.853689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb9182-7591-4534-88d7-dc1bb0466344 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:09.930801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5b7a6-5ff8-489b-bfe9-2259d3d92f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:09.931460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5b7a6-5ff8-489b-bfe9-2259d3d92f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:09.931773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5b7a6-5ff8-489b-bfe9-2259d3d92f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:09.932054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5b7a6-5ff8-489b-bfe9-2259d3d92f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:09.933786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3fb5b7a6-5ff8-489b-bfe9-2259d3d92f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:09.951236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:10.049392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5b7a6-5ff8-489b-bfe9-2259d3d92f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:10.050882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5b7a6-5ff8-489b-bfe9-2259d3d92f05 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:10.069294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 656/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:12.440199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b730-157d-4f3e-98b5-b3b722a89341 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:12.440759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b730-157d-4f3e-98b5-b3b722a89341 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:12.440759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360b730-157d-4f3e-98b5-b3b722a89341 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:12.442260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 673/1329] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:12.447696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e930341-15f8-46fc-bf4b-718cd72e55c5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:12.448173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e930341-15f8-46fc-bf4b-718cd72e55c5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:12.448283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e930341-15f8-46fc-bf4b-718cd72e55c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:12.518433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2e930341-15f8-46fc-bf4b-718cd72e55c5 None None] MFA Rules not processed for user `d695ee7d689a4ca4ae7d161f5773ca60`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:12.519700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2e930341-15f8-46fc-bf4b-718cd72e55c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:12.588218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:12.621377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 657/1330] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:12.665194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b730-157d-4f3e-98b5-b3b722a89341 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:12.666847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1360b730-157d-4f3e-98b5-b3b722a89341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:12.778252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360b730-157d-4f3e-98b5-b3b722a89341 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:12.801113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:12.857927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11b33f-206e-414e-a0cd-f88f755b6d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:12.859128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11b33f-206e-414e-a0cd-f88f755b6d91 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:12.859560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11b33f-206e-414e-a0cd-f88f755b6d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:12.860050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11b33f-206e-414e-a0cd-f88f755b6d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:12.861717 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1e11b33f-206e-414e-a0cd-f88f755b6d91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:12.976175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11b33f-206e-414e-a0cd-f88f755b6d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:12.977625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e11b33f-206e-414e-a0cd-f88f755b6d91 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:12.996699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 674/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:15.765519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0323ff-d816-4fb1-9870-71f3a19b96c0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:15.766283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0323ff-d816-4fb1-9870-71f3a19b96c0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:15.766283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0323ff-d816-4fb1-9870-71f3a19b96c0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:15.767484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 658/1332] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:15.773199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d67a2-4d1f-443f-8352-a20a6a0813d6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:15.773741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d67a2-4d1f-443f-8352-a20a6a0813d6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:15.773741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d67a2-4d1f-443f-8352-a20a6a0813d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:15.839800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:15.843388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-349d67a2-4d1f-443f-8352-a20a6a0813d6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:15.844642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-349d67a2-4d1f-443f-8352-a20a6a0813d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:15.958295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 675/1333] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:15:15.993703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0323ff-d816-4fb1-9870-71f3a19b96c0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:15.995247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5e0323ff-d816-4fb1-9870-71f3a19b96c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:16.111794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0323ff-d816-4fb1-9870-71f3a19b96c0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:16.195498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df270f-2e0d-4380-95e7-30033b3b9496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:16.196159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df270f-2e0d-4380-95e7-30033b3b9496 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:16.196450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df270f-2e0d-4380-95e7-30033b3b9496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:16.196727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df270f-2e0d-4380-95e7-30033b3b9496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:16.198387 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-52df270f-2e0d-4380-95e7-30033b3b9496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:16.228038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:16.340518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df270f-2e0d-4380-95e7-30033b3b9496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:16.341988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df270f-2e0d-4380-95e7-30033b3b9496 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:16.363963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 659/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:18.738684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399ca15-802b-41c1-9899-2fb0287d2f5b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:18.739257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399ca15-802b-41c1-9899-2fb0287d2f5b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:18.739257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399ca15-802b-41c1-9899-2fb0287d2f5b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:18.740658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 676/1335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:18.745841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d972f7-6ea2-4865-aa52-655f4c62ab9f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:18.746249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d972f7-6ea2-4865-aa52-655f4c62ab9f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:18.746426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d972f7-6ea2-4865-aa52-655f4c62ab9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:18.813032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-93d972f7-6ea2-4865-aa52-655f4c62ab9f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:18.814267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-93d972f7-6ea2-4865-aa52-655f4c62ab9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:18.887712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:18.910061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 660/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:18.950499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399ca15-802b-41c1-9899-2fb0287d2f5b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:18.952074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c399ca15-802b-41c1-9899-2fb0287d2f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:19.059022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c399ca15-802b-41c1-9899-2fb0287d2f5b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:19.088291 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:19.133317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e5d26-842c-4271-b9f7-28761c9b4f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:19.133969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e5d26-842c-4271-b9f7-28761c9b4f30 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:19.134256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e5d26-842c-4271-b9f7-28761c9b4f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:19.134573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e5d26-842c-4271-b9f7-28761c9b4f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:19.136256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e62e5d26-842c-4271-b9f7-28761c9b4f30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:19.245485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e5d26-842c-4271-b9f7-28761c9b4f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:19.246949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e5d26-842c-4271-b9f7-28761c9b4f30 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:19.267104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 677/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:22.318288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8b821-33ff-43be-9f88-54168371feb2 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:22.318844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8b821-33ff-43be-9f88-54168371feb2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:22.318844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8b821-33ff-43be-9f88-54168371feb2 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:22.320177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 661/1338] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:22.325581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e809a-9748-414a-a844-f6b5f966ad2b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:22.325990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e809a-9748-414a-a844-f6b5f966ad2b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:22.326231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e809a-9748-414a-a844-f6b5f966ad2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:22.400567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c14e809a-9748-414a-a844-f6b5f966ad2b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:22.401764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c14e809a-9748-414a-a844-f6b5f966ad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:22.411398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:22.511036 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 678/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:22.547517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8b821-33ff-43be-9f88-54168371feb2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:22.549633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a3f8b821-33ff-43be-9f88-54168371feb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:22.646899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8b821-33ff-43be-9f88-54168371feb2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:22.650396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44467c-a957-4910-8720-f1ac12bbd6e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:22.651062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44467c-a957-4910-8720-f1ac12bbd6e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:22.651346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44467c-a957-4910-8720-f1ac12bbd6e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:22.651628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44467c-a957-4910-8720-f1ac12bbd6e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:22.661140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44467c-a957-4910-8720-f1ac12bbd6e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:22.662610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44467c-a957-4910-8720-f1ac12bbd6e0 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:22.672417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 662/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:27.385474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac020c-f1e7-4eb5-8e59-ec4f888c8fad None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:27.386033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac020c-f1e7-4eb5-8e59-ec4f888c8fad None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:27.386258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac020c-f1e7-4eb5-8e59-ec4f888c8fad None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:27.387788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 679/1341] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:27.394175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79977542-59ed-4f27-ab46-fb5761ac03b9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:27.394945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79977542-59ed-4f27-ab46-fb5761ac03b9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:27.395117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79977542-59ed-4f27-ab46-fb5761ac03b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:27.431178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-79977542-59ed-4f27-ab46-fb5761ac03b9 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:27.432293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-79977542-59ed-4f27-ab46-fb5761ac03b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:27.526105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 663/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:27.566532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac020c-f1e7-4eb5-8e59-ec4f888c8fad None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:27.577858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac020c-f1e7-4eb5-8e59-ec4f888c8fad None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:27.581800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03367f-732a-4ec1-86a2-e501d72eb220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:27.582453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03367f-732a-4ec1-86a2-e501d72eb220 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:27.582742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03367f-732a-4ec1-86a2-e501d72eb220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:27.583007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03367f-732a-4ec1-86a2-e501d72eb220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:27.592587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03367f-732a-4ec1-86a2-e501d72eb220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:27.594242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03367f-732a-4ec1-86a2-e501d72eb220 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:27.603659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 680/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:30.963238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29baabf-945c-44b3-86e9-74332e4d9d5d None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:30.963895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29baabf-945c-44b3-86e9-74332e4d9d5d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:30.963895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29baabf-945c-44b3-86e9-74332e4d9d5d None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:30.965171 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 664/1344] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:30.971364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538faecf-1320-4c18-b3a5-fa077be200df None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:30.971871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538faecf-1320-4c18-b3a5-fa077be200df None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:30.971871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538faecf-1320-4c18-b3a5-fa077be200df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:30.991075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-538faecf-1320-4c18-b3a5-fa077be200df None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:30.992303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-538faecf-1320-4c18-b3a5-fa077be200df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:31.001055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 681/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:31.039307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29baabf-945c-44b3-86e9-74332e4d9d5d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:31.051073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29baabf-945c-44b3-86e9-74332e4d9d5d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:31.054802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15696363-1773-4a6c-94da-1122549c03ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:31.055449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696363-1773-4a6c-94da-1122549c03ad service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:31.055735 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696363-1773-4a6c-94da-1122549c03ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:31.056017 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15696363-1773-4a6c-94da-1122549c03ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:31.067953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696363-1773-4a6c-94da-1122549c03ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:31.069520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15696363-1773-4a6c-94da-1122549c03ad service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:31.080395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 665/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:35.747392 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d0e43-aed4-46ff-9a9f-85b444dc720a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:35.747933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d0e43-aed4-46ff-9a9f-85b444dc720a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:35.747933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d0e43-aed4-46ff-9a9f-85b444dc720a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:35.749371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 682/1347] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:35.754862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a910ddc-d0ce-4230-83f5-613bf0bf6974 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:35.755321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a910ddc-d0ce-4230-83f5-613bf0bf6974 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:35.755501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a910ddc-d0ce-4230-83f5-613bf0bf6974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:35.773944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9a910ddc-d0ce-4230-83f5-613bf0bf6974 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:35.775575 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9a910ddc-d0ce-4230-83f5-613bf0bf6974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:35.783823 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 666/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:35.821354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d0e43-aed4-46ff-9a9f-85b444dc720a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:35.832922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d0e43-aed4-46ff-9a9f-85b444dc720a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:35.836551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fce91-f5d0-4d8f-8595-f642777643e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:35.837264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fce91-f5d0-4d8f-8595-f642777643e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:35.837616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fce91-f5d0-4d8f-8595-f642777643e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:35.837962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fce91-f5d0-4d8f-8595-f642777643e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:35.848180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fce91-f5d0-4d8f-8595-f642777643e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:35.849620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fce91-f5d0-4d8f-8595-f642777643e9 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:35.859345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 683/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:41.452373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cb68-b458-4636-9fa6-47073216d2ac None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:41.453285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cb68-b458-4636-9fa6-47073216d2ac None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:41.453781 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2cb68-b458-4636-9fa6-47073216d2ac None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:41.455702 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 667/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:41.460853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7672f-6f85-475a-a735-6f8b37a39292 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:41.461268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7672f-6f85-475a-a735-6f8b37a39292 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:41.461362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7672f-6f85-475a-a735-6f8b37a39292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:41.637908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-94a7672f-6f85-475a-a735-6f8b37a39292 None None] MFA Rules not processed for user `4f41920bad3147bba0000de911dfe5a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:41.639716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-94a7672f-6f85-475a-a735-6f8b37a39292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:41.800340 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:41.845936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 684/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:15:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:45.769552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:15:46.147837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3487d-6988-433c-a943-6075fdd71ec5 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:46.148159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3487d-6988-433c-a943-6075fdd71ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:46.148548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3487d-6988-433c-a943-6075fdd71ec5 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:46.149931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 668/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:46.155511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622921-e96a-411e-b76e-cc99cf927a99 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:46.156623 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622921-e96a-411e-b76e-cc99cf927a99 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:46.157145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622921-e96a-411e-b76e-cc99cf927a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:46.182548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3b622921-e96a-411e-b76e-cc99cf927a99 None None] MFA Rules not processed for user `4f41920bad3147bba0000de911dfe5a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:46.184306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3b622921-e96a-411e-b76e-cc99cf927a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:46.196945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 685/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:15:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:47.835955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd3d79-f70c-4442-9f9c-af000779ffa8 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:47.836537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd3d79-f70c-4442-9f9c-af000779ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:47.836537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd3d79-f70c-4442-9f9c-af000779ffa8 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:47.838015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 669/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:47.844287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b66a9-2da0-48b0-bdd5-a1a2ed8a3fbc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:47.844759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b66a9-2da0-48b0-bdd5-a1a2ed8a3fbc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:47.844901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b66a9-2da0-48b0-bdd5-a1a2ed8a3fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:47.863432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2c0b66a9-2da0-48b0-bdd5-a1a2ed8a3fbc None None] MFA Rules not processed for user `4f41920bad3147bba0000de911dfe5a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:47.864643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2c0b66a9-2da0-48b0-bdd5-a1a2ed8a3fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:47.873431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 686/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:15:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:51.884955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49225512-0598-4ef4-a497-f2cd0d043da2 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:51.885488 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49225512-0598-4ef4-a497-f2cd0d043da2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:51.885488 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49225512-0598-4ef4-a497-f2cd0d043da2 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:51.887185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 670/1356] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:51.893544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dac5cf-bbf4-4f8c-a2f7-a4c21a5b979f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:51.894012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dac5cf-bbf4-4f8c-a2f7-a4c21a5b979f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:51.894105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dac5cf-bbf4-4f8c-a2f7-a4c21a5b979f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:51.913070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d7dac5cf-bbf4-4f8c-a2f7-a4c21a5b979f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:51.914800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d7dac5cf-bbf4-4f8c-a2f7-a4c21a5b979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:51.924376 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 687/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:51.986885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73762b-846d-465a-9eb1-8df5a553b67a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:51.987074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73762b-846d-465a-9eb1-8df5a553b67a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:51.987357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73762b-846d-465a-9eb1-8df5a553b67a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:51.988712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 671/1358] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 12:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:51.999003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a24e8-b7a0-4430-ad8d-295d8f47a421 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:51.999329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a24e8-b7a0-4430-ad8d-295d8f47a421 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:51.999747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a24e8-b7a0-4430-ad8d-295d8f47a421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:52.064395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-171a24e8-b7a0-4430-ad8d-295d8f47a421 None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:52.065840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-171a24e8-b7a0-4430-ad8d-295d8f47a421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:52.159276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 688/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:52.167490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73762b-846d-465a-9eb1-8df5a553b67a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:52.182801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73762b-846d-465a-9eb1-8df5a553b67a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:52.188747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fc53a-d9eb-46e3-8e50-8b625209a2e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:52.189381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fc53a-d9eb-46e3-8e50-8b625209a2e1 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:52.189663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fc53a-d9eb-46e3-8e50-8b625209a2e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:52.189941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fc53a-d9eb-46e3-8e50-8b625209a2e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:52.198848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fc53a-d9eb-46e3-8e50-8b625209a2e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:52.200512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fc53a-d9eb-46e3-8e50-8b625209a2e1 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:52.210636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 672/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:54.707205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729df587-b044-47fa-aa6c-696778e97361 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:54.707664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729df587-b044-47fa-aa6c-696778e97361 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:54.707836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729df587-b044-47fa-aa6c-696778e97361 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:54.713312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 689/1361] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:54.715282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf9327-5ee1-4977-992d-482b3e23876e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:54.716729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf9327-5ee1-4977-992d-482b3e23876e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:54.717222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf9327-5ee1-4977-992d-482b3e23876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:54.736777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-afdf9327-5ee1-4977-992d-482b3e23876e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:54.738347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-afdf9327-5ee1-4977-992d-482b3e23876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:54.746572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 673/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:15:54.793032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729df587-b044-47fa-aa6c-696778e97361 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:54.811050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729df587-b044-47fa-aa6c-696778e97361 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:54.817300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf92ba-057d-44ce-9f36-e8f051136252 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:54.818271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf92ba-057d-44ce-9f36-e8f051136252 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:54.818610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf92ba-057d-44ce-9f36-e8f051136252 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:54.818893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf92ba-057d-44ce-9f36-e8f051136252 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:54.829503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf92ba-057d-44ce-9f36-e8f051136252 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:54.831182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf92ba-057d-44ce-9f36-e8f051136252 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:54.841847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 690/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:15:57.807909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46762f-8ce2-41d4-a12f-520df012a3c9 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:57.808451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46762f-8ce2-41d4-a12f-520df012a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:57.808548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46762f-8ce2-41d4-a12f-520df012a3c9 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:57.810184 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 674/1364] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:57.815763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61811d-025b-421d-8e11-d0251c507e41 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:57.816254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61811d-025b-421d-8e11-d0251c507e41 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:57.816372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61811d-025b-421d-8e11-d0251c507e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:57.836783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-fd61811d-025b-421d-8e11-d0251c507e41 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:57.837977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fd61811d-025b-421d-8e11-d0251c507e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:57.860306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 691/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:57.908032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3e33d-fdb3-4002-ab0a-38b0f3a7ffec None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:57.908310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3e33d-fdb3-4002-ab0a-38b0f3a7ffec None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:57.908590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3e33d-fdb3-4002-ab0a-38b0f3a7ffec None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:57.909949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 675/1366] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 12:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:15:57.920170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0f23a-b537-4e65-a59b-19d27345ab69 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:57.920170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0f23a-b537-4e65-a59b-19d27345ab69 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:57.920170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0f23a-b537-4e65-a59b-19d27345ab69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:57.939429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e7a0f23a-b537-4e65-a59b-19d27345ab69 None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:15:57.940642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e7a0f23a-b537-4e65-a59b-19d27345ab69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:15:57.950954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 692/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 12:15:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:15:57.964303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3e33d-fdb3-4002-ab0a-38b0f3a7ffec None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:15:57.980044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3e33d-fdb3-4002-ab0a-38b0f3a7ffec None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:15:57.997777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a511f83-d419-47d3-9cca-0ed3ffdb1683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:15:57.998444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a511f83-d419-47d3-9cca-0ed3ffdb1683 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:15:57.998734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a511f83-d419-47d3-9cca-0ed3ffdb1683 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:15:57.999012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a511f83-d419-47d3-9cca-0ed3ffdb1683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:15:58.018423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a511f83-d419-47d3-9cca-0ed3ffdb1683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:15:58.021621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a511f83-d419-47d3-9cca-0ed3ffdb1683 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:15:58.035583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 676/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:16:00.635233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf0f44-7ede-47ff-9ad2-1d6be041eb46 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:00.636077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf0f44-7ede-47ff-9ad2-1d6be041eb46 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:00.636077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf0f44-7ede-47ff-9ad2-1d6be041eb46 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:00.637231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 693/1369] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:00.644182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c9e11-605c-400a-a119-7478fa607ef5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:00.644763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c9e11-605c-400a-a119-7478fa607ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:00.644763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c9e11-605c-400a-a119-7478fa607ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:00.667044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-de3c9e11-605c-400a-a119-7478fa607ef5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:00.668857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-de3c9e11-605c-400a-a119-7478fa607ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:00.678770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 677/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:00.688832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2678c-7a81-4875-8db0-0b4c74c7092b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:00.689116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2678c-7a81-4875-8db0-0b4c74c7092b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:00.689581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2678c-7a81-4875-8db0-0b4c74c7092b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:00.690991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 694/1371] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:00.696825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6e518-31f0-414c-9b92-3357dc81ebb7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:00.697168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6e518-31f0-414c-9b92-3357dc81ebb7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:00.697386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6e518-31f0-414c-9b92-3357dc81ebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:00.718042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c6e6e518-31f0-414c-9b92-3357dc81ebb7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:00.719372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c6e6e518-31f0-414c-9b92-3357dc81ebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:00.727739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 678/1372] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:00.775728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2678c-7a81-4875-8db0-0b4c74c7092b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:00.791121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2678c-7a81-4875-8db0-0b4c74c7092b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:00.797260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bad0e-2db8-413c-9073-e44e7243d172 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:00.797905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bad0e-2db8-413c-9073-e44e7243d172 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:00.798364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bad0e-2db8-413c-9073-e44e7243d172 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:00.798825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bad0e-2db8-413c-9073-e44e7243d172 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:00.807878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bad0e-2db8-413c-9073-e44e7243d172 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:00.810230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bad0e-2db8-413c-9073-e44e7243d172 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:00.825766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 695/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:00.963850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadddf1-2845-4d1d-95b3-97e7b0442155 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:00.964148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadddf1-2845-4d1d-95b3-97e7b0442155 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:00.964434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feadddf1-2845-4d1d-95b3-97e7b0442155 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:00.965867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 679/1374] 10.222.0.57 () {54 vars in 963 bytes} [Thu May 19 12:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:00.971523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b283f-693a-4f63-b048-893edc29724f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:00.971821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b283f-693a-4f63-b048-893edc29724f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:00.972208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b283f-693a-4f63-b048-893edc29724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:01.036570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c45b283f-693a-4f63-b048-893edc29724f None None] MFA Rules not processed for user `316ad1c2eccf4642a21c68856585ceee`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:01.037928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c45b283f-693a-4f63-b048-893edc29724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:01.051864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:01.121048 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 696/1375] 10.222.0.57 () {58 vars in 1060 bytes} [Thu May 19 12:16:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:01.130129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadddf1-2845-4d1d-95b3-97e7b0442155 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:01.142875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feadddf1-2845-4d1d-95b3-97e7b0442155 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:01.147598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74c1f9-d73a-498f-bca5-d61a1f912048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:01.148450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74c1f9-d73a-498f-bca5-d61a1f912048 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:01.148822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74c1f9-d73a-498f-bca5-d61a1f912048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:01.149171 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74c1f9-d73a-498f-bca5-d61a1f912048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:01.160992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74c1f9-d73a-498f-bca5-d61a1f912048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:01.162680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74c1f9-d73a-498f-bca5-d61a1f912048 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:01.176533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 680/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:03.322140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89724fa9-94aa-4cc7-8d3d-57f268846c12 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:03.322680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89724fa9-94aa-4cc7-8d3d-57f268846c12 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:03.322680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89724fa9-94aa-4cc7-8d3d-57f268846c12 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:03.323992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 697/1377] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:03.330728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9998717-d92a-418c-b07d-5604bf1ec6f0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:03.331245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9998717-d92a-418c-b07d-5604bf1ec6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:03.331340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9998717-d92a-418c-b07d-5604bf1ec6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:03.349768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e9998717-d92a-418c-b07d-5604bf1ec6f0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:03.351010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e9998717-d92a-418c-b07d-5604bf1ec6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:03.359623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 681/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:16:03.405512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89724fa9-94aa-4cc7-8d3d-57f268846c12 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:03.421469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89724fa9-94aa-4cc7-8d3d-57f268846c12 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:03.428655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa562b-288c-45b3-a631-fc92a839cdb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:03.429322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa562b-288c-45b3-a631-fc92a839cdb2 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:03.429671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa562b-288c-45b3-a631-fc92a839cdb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:03.429883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa562b-288c-45b3-a631-fc92a839cdb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:03.440072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa562b-288c-45b3-a631-fc92a839cdb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:03.441193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa562b-288c-45b3-a631-fc92a839cdb2 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:03.451446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 698/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:05.957962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acea9bd-75a5-49ab-9aa9-90d2b47916c2 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:05.958663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acea9bd-75a5-49ab-9aa9-90d2b47916c2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:05.958663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acea9bd-75a5-49ab-9aa9-90d2b47916c2 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:05.959892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 682/1380] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:05.966243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aea58-efd2-4dc9-86ae-093f21f9f99a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:05.966779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aea58-efd2-4dc9-86ae-093f21f9f99a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:05.966883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496aea58-efd2-4dc9-86ae-093f21f9f99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:05.985233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-496aea58-efd2-4dc9-86ae-093f21f9f99a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:05.986648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-496aea58-efd2-4dc9-86ae-093f21f9f99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:05.994806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 699/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:06.040115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acea9bd-75a5-49ab-9aa9-90d2b47916c2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:06.055442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acea9bd-75a5-49ab-9aa9-90d2b47916c2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:06.061966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5f042-b502-4644-a2e9-9ed34d12c064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:06.062739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5f042-b502-4644-a2e9-9ed34d12c064 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:06.063082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5f042-b502-4644-a2e9-9ed34d12c064 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:06.067168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5f042-b502-4644-a2e9-9ed34d12c064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:06.072394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5f042-b502-4644-a2e9-9ed34d12c064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:06.074165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5f042-b502-4644-a2e9-9ed34d12c064 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:06.085998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 683/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:09.057238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5596d0d-d984-4626-adf3-8a5700d911a3 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:09.060708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5596d0d-d984-4626-adf3-8a5700d911a3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:09.060708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5596d0d-d984-4626-adf3-8a5700d911a3 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:09.060708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 700/1383] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:09.064865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01fa0c-3f7a-4428-887a-b05deaed698a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:09.065651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01fa0c-3f7a-4428-887a-b05deaed698a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:09.066093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01fa0c-3f7a-4428-887a-b05deaed698a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:09.087228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7f01fa0c-3f7a-4428-887a-b05deaed698a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:09.094299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7f01fa0c-3f7a-4428-887a-b05deaed698a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:09.098793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 684/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:09.141494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5596d0d-d984-4626-adf3-8a5700d911a3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:09.160176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5596d0d-d984-4626-adf3-8a5700d911a3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:09.166818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd5bfe-0268-4329-975d-d05cfde28f58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:09.167699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd5bfe-0268-4329-975d-d05cfde28f58 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:09.168304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd5bfe-0268-4329-975d-d05cfde28f58 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:09.169285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd5bfe-0268-4329-975d-d05cfde28f58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:09.180839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd5bfe-0268-4329-975d-d05cfde28f58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:09.184043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd5bfe-0268-4329-975d-d05cfde28f58 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:09.197993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 701/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:11.780594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430d6ec-7a0e-4096-ad8c-2a2c55c986a0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:11.781333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430d6ec-7a0e-4096-ad8c-2a2c55c986a0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:11.781333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430d6ec-7a0e-4096-ad8c-2a2c55c986a0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:11.783800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 685/1386] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:11.791006 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75412b-bf4e-41a5-9269-ab789f0f0365 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:11.791731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75412b-bf4e-41a5-9269-ab789f0f0365 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:11.791731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75412b-bf4e-41a5-9269-ab789f0f0365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:11.824507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3b75412b-bf4e-41a5-9269-ab789f0f0365 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:11.827919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3b75412b-bf4e-41a5-9269-ab789f0f0365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:11.845262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 702/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:11.898631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430d6ec-7a0e-4096-ad8c-2a2c55c986a0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:11.920035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430d6ec-7a0e-4096-ad8c-2a2c55c986a0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:11.930994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a6cb3-36be-4a9b-b879-1e4e1ede1e54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:11.931659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a6cb3-36be-4a9b-b879-1e4e1ede1e54 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:11.931944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a6cb3-36be-4a9b-b879-1e4e1ede1e54 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:11.932168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a6cb3-36be-4a9b-b879-1e4e1ede1e54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:11.944738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a6cb3-36be-4a9b-b879-1e4e1ede1e54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:11.946479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a6cb3-36be-4a9b-b879-1e4e1ede1e54 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:11.960346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 686/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:15.559097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe9e5c-ae77-4b5e-8d87-2444377ab883 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:15.559803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe9e5c-ae77-4b5e-8d87-2444377ab883 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:15.559803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe9e5c-ae77-4b5e-8d87-2444377ab883 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:15.561093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 703/1389] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:16:15.568582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02713ce6-7ef0-43a8-b7a6-4ecb668a97ca None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:15.569288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02713ce6-7ef0-43a8-b7a6-4ecb668a97ca None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:15.569288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02713ce6-7ef0-43a8-b7a6-4ecb668a97ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:15.593937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-02713ce6-7ef0-43a8-b7a6-4ecb668a97ca None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:15.595491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-02713ce6-7ef0-43a8-b7a6-4ecb668a97ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:15.606046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 687/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:15.660748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe9e5c-ae77-4b5e-8d87-2444377ab883 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:15.676868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe9e5c-ae77-4b5e-8d87-2444377ab883 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:15.682449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459be8f-0bf3-4f24-a747-10a2d15cd691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:15.683112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459be8f-0bf3-4f24-a747-10a2d15cd691 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:15.683431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459be8f-0bf3-4f24-a747-10a2d15cd691 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:15.683706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459be8f-0bf3-4f24-a747-10a2d15cd691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:15.694633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459be8f-0bf3-4f24-a747-10a2d15cd691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:15.696300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459be8f-0bf3-4f24-a747-10a2d15cd691 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:15.710815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 704/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:18.227537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27416ab6-d0bd-45f2-8eee-b63658a4e061 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:18.228136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27416ab6-d0bd-45f2-8eee-b63658a4e061 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:18.228136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27416ab6-d0bd-45f2-8eee-b63658a4e061 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:18.229447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 688/1392] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:18.235329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8718f-eaea-4a22-ae42-be38e148d40d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:18.235845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8718f-eaea-4a22-ae42-be38e148d40d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:18.235929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8718f-eaea-4a22-ae42-be38e148d40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:18.258234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7ed8718f-eaea-4a22-ae42-be38e148d40d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:18.258234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7ed8718f-eaea-4a22-ae42-be38e148d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:18.271529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 705/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:18.331180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9321475-3dd8-4f74-9629-ed5ef532d7b0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:18.331476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9321475-3dd8-4f74-9629-ed5ef532d7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:18.331760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9321475-3dd8-4f74-9629-ed5ef532d7b0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:18.333145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 689/1394] 10.222.0.57 () {54 vars in 962 bytes} [Thu May 19 12:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:18.338514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f491a5a-21b6-4467-8acc-7e436ae1e1e8 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:18.338814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f491a5a-21b6-4467-8acc-7e436ae1e1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:18.339099 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f491a5a-21b6-4467-8acc-7e436ae1e1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:18.395640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7f491a5a-21b6-4467-8acc-7e436ae1e1e8 None None] MFA Rules not processed for user `9969632e796345fa9671a45cf61ede3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:18.396870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7f491a5a-21b6-4467-8acc-7e436ae1e1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:18.460600 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 706/1395] 10.222.0.57 () {58 vars in 1059 bytes} [Thu May 19 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:18.468141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9321475-3dd8-4f74-9629-ed5ef532d7b0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:18.473358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=49 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:18.479433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9321475-3dd8-4f74-9629-ed5ef532d7b0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:18.483912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8d5e3-20fb-4a73-ae8e-b30ae22d953d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9969632e796345fa9671a45cf61ede3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:18.484724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8d5e3-20fb-4a73-ae8e-b30ae22d953d service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:18.485068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8d5e3-20fb-4a73-ae8e-b30ae22d953d service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:18.485378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8d5e3-20fb-4a73-ae8e-b30ae22d953d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:18.495190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8d5e3-20fb-4a73-ae8e-b30ae22d953d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:18.497774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8d5e3-20fb-4a73-ae8e-b30ae22d953d service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:18.511790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 690/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:18.563543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5d775-9302-4000-8c7b-15fac28a6891 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:18.563806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5d775-9302-4000-8c7b-15fac28a6891 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:18.564040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5d775-9302-4000-8c7b-15fac28a6891 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:18.569281 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 707/1397] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:18.571428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2539c13-0e25-4519-a4af-70fe7a6a854a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:18.571738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2539c13-0e25-4519-a4af-70fe7a6a854a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:18.572022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2539c13-0e25-4519-a4af-70fe7a6a854a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:18.593111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c2539c13-0e25-4519-a4af-70fe7a6a854a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:18.594327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c2539c13-0e25-4519-a4af-70fe7a6a854a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:18.610384 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 691/1398] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:18.650908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5d775-9302-4000-8c7b-15fac28a6891 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:18.670462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5d775-9302-4000-8c7b-15fac28a6891 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:18.677758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef585664-c474-4bbc-9688-72b26af9f94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:18.678493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef585664-c474-4bbc-9688-72b26af9f94b service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:18.678741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef585664-c474-4bbc-9688-72b26af9f94b service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:18.679028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef585664-c474-4bbc-9688-72b26af9f94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:18.689723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef585664-c474-4bbc-9688-72b26af9f94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:18.691388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef585664-c474-4bbc-9688-72b26af9f94b service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:18.703058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 708/1399] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:19.305891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0b564-15bd-4e5a-b75d-70c0ca3713c4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:19.306422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0b564-15bd-4e5a-b75d-70c0ca3713c4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:19.306579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0b564-15bd-4e5a-b75d-70c0ca3713c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:19.360513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-64b0b564-15bd-4e5a-b75d-70c0ca3713c4 None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:19.361700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-64b0b564-15bd-4e5a-b75d-70c0ca3713c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:19.430445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 692/1400] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:16:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:19.475190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3a18a-561b-4d2c-b473-563de407e45c None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:19.475746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3a18a-561b-4d2c-b473-563de407e45c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:19.475840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3a18a-561b-4d2c-b473-563de407e45c None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:19.477612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 709/1401] 10.222.0.57 () {54 vars in 942 bytes} [Thu May 19 12:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:19.484683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44fac0-d578-4f47-94ef-f24a5916a058 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:19.484991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44fac0-d578-4f47-94ef-f24a5916a058 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:19.485279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44fac0-d578-4f47-94ef-f24a5916a058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:19.537686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6e44fac0-d578-4f47-94ef-f24a5916a058 None None] MFA Rules not processed for user `28ddf898ad7b4e10aaf0770249b4453d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:19.539120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6e44fac0-d578-4f47-94ef-f24a5916a058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:19.615047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 693/1402] 10.222.0.57 () {58 vars in 1039 bytes} [Thu May 19 12:16:19 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:19.625000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3a18a-561b-4d2c-b473-563de407e45c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:19.638853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3a18a-561b-4d2c-b473-563de407e45c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:19.643776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcc264-9031-4afa-88f5-460d5b824c7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:19.644422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcc264-9031-4afa-88f5-460d5b824c7f service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:19.644707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcc264-9031-4afa-88f5-460d5b824c7f service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:19.644990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcc264-9031-4afa-88f5-460d5b824c7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:19.653687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcc264-9031-4afa-88f5-460d5b824c7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:19.655305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefcc264-9031-4afa-88f5-460d5b824c7f service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:19.664064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 710/1403] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:16:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:16:21.231809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4d772-857b-4072-a4c1-978af4a1bbfd None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:21.232369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4d772-857b-4072-a4c1-978af4a1bbfd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:21.232369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4d772-857b-4072-a4c1-978af4a1bbfd None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:21.233755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 694/1404] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:21.239427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f702e-173d-4019-8427-3461ebdc20dd None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:21.240107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f702e-173d-4019-8427-3461ebdc20dd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:21.240107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f702e-173d-4019-8427-3461ebdc20dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:21.252880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:21.259120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2b7f702e-173d-4019-8427-3461ebdc20dd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:21.260353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2b7f702e-173d-4019-8427-3461ebdc20dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:21.269748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 711/1405] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:16:21.317760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4d772-857b-4072-a4c1-978af4a1bbfd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:21.338618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4d772-857b-4072-a4c1-978af4a1bbfd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:21.345188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43801b7-8a07-4fec-97ca-75fb3f433a40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:21.345901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43801b7-8a07-4fec-97ca-75fb3f433a40 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:21.346208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43801b7-8a07-4fec-97ca-75fb3f433a40 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:21.346525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43801b7-8a07-4fec-97ca-75fb3f433a40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:21.356106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43801b7-8a07-4fec-97ca-75fb3f433a40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:21.357857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43801b7-8a07-4fec-97ca-75fb3f433a40 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:21.368981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 695/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:24.884966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760a5f-5834-4ce0-9948-d03c37662cba None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:24.885535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760a5f-5834-4ce0-9948-d03c37662cba None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:24.885535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760a5f-5834-4ce0-9948-d03c37662cba None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:24.887003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 712/1407] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:24.892518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7e407-b0aa-41a6-8210-f45369274058 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:24.893063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7e407-b0aa-41a6-8210-f45369274058 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:24.893063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7e407-b0aa-41a6-8210-f45369274058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:24.900598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:24.912849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-39d7e407-b0aa-41a6-8210-f45369274058 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:24.914223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-39d7e407-b0aa-41a6-8210-f45369274058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:24.924422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 696/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:24.976334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760a5f-5834-4ce0-9948-d03c37662cba None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:24.992233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760a5f-5834-4ce0-9948-d03c37662cba None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:24.998299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b72a7b-02ef-47f2-8998-7004d646ffea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:24.999071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b72a7b-02ef-47f2-8998-7004d646ffea service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:24.999362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b72a7b-02ef-47f2-8998-7004d646ffea service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:24.999701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b72a7b-02ef-47f2-8998-7004d646ffea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:25.009378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b72a7b-02ef-47f2-8998-7004d646ffea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:25.011159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b72a7b-02ef-47f2-8998-7004d646ffea service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:25.022276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 713/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:34.942828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:35.017101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=6 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:16:44.801825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfe463-7d2c-4bb3-bccb-9553bb889a66 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:44.802478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfe463-7d2c-4bb3-bccb-9553bb889a66 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:44.802478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfe463-7d2c-4bb3-bccb-9553bb889a66 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:44.803827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 697/1410] 10.222.0.57 () {54 vars in 918 bytes} [Thu May 19 12:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:44.809016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f51f5-cb9e-4bac-8753-97cb6763e429 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:44.809431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f51f5-cb9e-4bac-8753-97cb6763e429 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:44.809575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f51f5-cb9e-4bac-8753-97cb6763e429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:44.828144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cc0f51f5-cb9e-4bac-8753-97cb6763e429 None None] MFA Rules not processed for user `4f41920bad3147bba0000de911dfe5a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:44.830318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cc0f51f5-cb9e-4bac-8753-97cb6763e429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:44.838780 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 714/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Thu May 19 12:16:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:44.853070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110173e-b4bf-4cbd-8a7d-27abe556c072 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:44.853358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110173e-b4bf-4cbd-8a7d-27abe556c072 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:44.853638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110173e-b4bf-4cbd-8a7d-27abe556c072 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:44.855098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 698/1412] 10.222.0.57 () {54 vars in 918 bytes} [Thu May 19 12:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:44.860450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107849e4-92d7-486a-9551-990a580d3318 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:44.860839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107849e4-92d7-486a-9551-990a580d3318 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:44.861325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107849e4-92d7-486a-9551-990a580d3318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:44.881070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-107849e4-92d7-486a-9551-990a580d3318 None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:44.882254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-107849e4-92d7-486a-9551-990a580d3318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:44.891280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 715/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Thu May 19 12:16:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:47.601885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55372448-8208-446e-b72e-0ab2e26aa683 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:47.602498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55372448-8208-446e-b72e-0ab2e26aa683 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:47.602498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55372448-8208-446e-b72e-0ab2e26aa683 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:47.603896 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:16:47.609431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c3212-7a98-4569-bb9e-2c3b0fdb0bf4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:47.610039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c3212-7a98-4569-bb9e-2c3b0fdb0bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:47.610564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c3212-7a98-4569-bb9e-2c3b0fdb0bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:47.757625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-453c3212-7a98-4569-bb9e-2c3b0fdb0bf4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:47.758774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-453c3212-7a98-4569-bb9e-2c3b0fdb0bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:47.767636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:47.810357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55372448-8208-446e-b72e-0ab2e26aa683 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:47.825237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55372448-8208-446e-b72e-0ab2e26aa683 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:47.831638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11ea17-d17d-49dc-96c2-ca70a9a3d705 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:47.832304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11ea17-d17d-49dc-96c2-ca70a9a3d705 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:47.832588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11ea17-d17d-49dc-96c2-ca70a9a3d705 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:47.832865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11ea17-d17d-49dc-96c2-ca70a9a3d705 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:47.841529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11ea17-d17d-49dc-96c2-ca70a9a3d705 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:47.843143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11ea17-d17d-49dc-96c2-ca70a9a3d705 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:47.853132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 700/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:48.856804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbbcd7-bc32-4a90-a33b-6e4f65b097f0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:48.857423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbbcd7-bc32-4a90-a33b-6e4f65b097f0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:48.857873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbbcd7-bc32-4a90-a33b-6e4f65b097f0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:48.860417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:16:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:48.865847 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b527c-8478-432f-8b6d-647021309e9f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:48.866376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b527c-8478-432f-8b6d-647021309e9f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:48.866503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b527c-8478-432f-8b6d-647021309e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:48.889583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-132b527c-8478-432f-8b6d-647021309e9f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:48.890836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-132b527c-8478-432f-8b6d-647021309e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:48.899327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:49.084457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbbcd7-bc32-4a90-a33b-6e4f65b097f0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:49.098895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbbcd7-bc32-4a90-a33b-6e4f65b097f0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:49.104379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d639940-3c00-486d-b533-ccdef84d2ac6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:49.105028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639940-3c00-486d-b533-ccdef84d2ac6 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:49.105313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639940-3c00-486d-b533-ccdef84d2ac6 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:49.105595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d639940-3c00-486d-b533-ccdef84d2ac6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:49.114469 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d639940-3c00-486d-b533-ccdef84d2ac6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:49.116044 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d639940-3c00-486d-b533-ccdef84d2ac6 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:49.125261 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 718/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:50.097193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce075e80-a3fc-4dd5-b780-a2d5557c9d7e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:50.097755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce075e80-a3fc-4dd5-b780-a2d5557c9d7e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:50.097755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce075e80-a3fc-4dd5-b780-a2d5557c9d7e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:50.099088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:50.104140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507013fa-d477-413e-80bd-6efae4c1d271 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:50.108268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507013fa-d477-413e-80bd-6efae4c1d271 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:50.108268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507013fa-d477-413e-80bd-6efae4c1d271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:50.123339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-507013fa-d477-413e-80bd-6efae4c1d271 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:50.124570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-507013fa-d477-413e-80bd-6efae4c1d271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:50.132505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:50.288911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce075e80-a3fc-4dd5-b780-a2d5557c9d7e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:50.312556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce075e80-a3fc-4dd5-b780-a2d5557c9d7e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:50.318130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b56c7-86d4-4e9b-a505-801dfcca3ac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:50.318792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b56c7-86d4-4e9b-a505-801dfcca3ac7 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:50.319075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b56c7-86d4-4e9b-a505-801dfcca3ac7 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:50.319392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b56c7-86d4-4e9b-a505-801dfcca3ac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:50.327994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b56c7-86d4-4e9b-a505-801dfcca3ac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:50.329687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b56c7-86d4-4e9b-a505-801dfcca3ac7 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:50.339328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 703/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:51.282187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c94aad-501e-487e-84cf-fc3464a2b329 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:51.283035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c94aad-501e-487e-84cf-fc3464a2b329 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:51.283035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c94aad-501e-487e-84cf-fc3464a2b329 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:51.284088 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:51.289239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520d1d6-9110-429f-9699-a2f514405d7f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:51.289692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520d1d6-9110-429f-9699-a2f514405d7f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:51.289806 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520d1d6-9110-429f-9699-a2f514405d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:51.308073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e520d1d6-9110-429f-9699-a2f514405d7f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:51.309297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e520d1d6-9110-429f-9699-a2f514405d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:51.317047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:51.468467 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c94aad-501e-487e-84cf-fc3464a2b329 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:51.483428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c94aad-501e-487e-84cf-fc3464a2b329 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:51.489162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3260f-541b-448b-840b-388183a1e5a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:51.489785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3260f-541b-448b-840b-388183a1e5a0 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:51.490065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3260f-541b-448b-840b-388183a1e5a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:51.490337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3260f-541b-448b-840b-388183a1e5a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:51.498605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3260f-541b-448b-840b-388183a1e5a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:51.500195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3260f-541b-448b-840b-388183a1e5a0 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:51.509954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 721/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:52.464034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38080749-7c64-4b38-9546-bb61479942c2 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:52.464648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38080749-7c64-4b38-9546-bb61479942c2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:52.464648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38080749-7c64-4b38-9546-bb61479942c2 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:52.465962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:52.471061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214932f-b0f6-4558-9e7d-d0b718562ecc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:52.471481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214932f-b0f6-4558-9e7d-d0b718562ecc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:52.471630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214932f-b0f6-4558-9e7d-d0b718562ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:52.489270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6214932f-b0f6-4558-9e7d-d0b718562ecc None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:52.490720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6214932f-b0f6-4558-9e7d-d0b718562ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:52.499010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:52.649980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38080749-7c64-4b38-9546-bb61479942c2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:52.664850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38080749-7c64-4b38-9546-bb61479942c2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:52.670145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a5959-443f-4791-9eee-59cd1e47d5d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:52.670821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a5959-443f-4791-9eee-59cd1e47d5d6 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:52.671377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a5959-443f-4791-9eee-59cd1e47d5d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:52.671655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a5959-443f-4791-9eee-59cd1e47d5d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:52.679891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a5959-443f-4791-9eee-59cd1e47d5d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:52.681466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a5959-443f-4791-9eee-59cd1e47d5d6 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:52.691430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 706/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:16:53.665594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41849407-4895-4b50-a122-0bb952e97eb0 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:53.666141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41849407-4895-4b50-a122-0bb952e97eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:53.666141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41849407-4895-4b50-a122-0bb952e97eb0 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:53.667654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:53.673412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8bada-682e-47b4-8416-e912ee5f6175 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:53.675182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8bada-682e-47b4-8416-e912ee5f6175 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:53.675182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8bada-682e-47b4-8416-e912ee5f6175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:53.692405 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1ef8bada-682e-47b4-8416-e912ee5f6175 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:53.693644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1ef8bada-682e-47b4-8416-e912ee5f6175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:53.701410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:53.868399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41849407-4895-4b50-a122-0bb952e97eb0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:53.884325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41849407-4895-4b50-a122-0bb952e97eb0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:53.889660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071e07c-26e5-4869-b2f1-9f9d1b72fbae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:53.890294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071e07c-26e5-4869-b2f1-9f9d1b72fbae service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:53.890606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071e07c-26e5-4869-b2f1-9f9d1b72fbae service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:53.891105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071e07c-26e5-4869-b2f1-9f9d1b72fbae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:53.900251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071e07c-26e5-4869-b2f1-9f9d1b72fbae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:53.901818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071e07c-26e5-4869-b2f1-9f9d1b72fbae service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:53.913147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 724/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:54.843177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73243b9-4abd-4e79-ab67-a8ee1eefdd63 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:54.843708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73243b9-4abd-4e79-ab67-a8ee1eefdd63 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:54.844365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73243b9-4abd-4e79-ab67-a8ee1eefdd63 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:54.846032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:54.851123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed60c94-5805-4ba9-8269-6f15adb4ffad None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:54.851597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed60c94-5805-4ba9-8269-6f15adb4ffad None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:54.851693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed60c94-5805-4ba9-8269-6f15adb4ffad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:54.870175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3ed60c94-5805-4ba9-8269-6f15adb4ffad None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:54.871445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3ed60c94-5805-4ba9-8269-6f15adb4ffad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:54.879599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:55.030252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73243b9-4abd-4e79-ab67-a8ee1eefdd63 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:55.045026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73243b9-4abd-4e79-ab67-a8ee1eefdd63 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:55.050750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d963b-0556-4520-8cb7-8e4a2c7f6277 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:55.051377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d963b-0556-4520-8cb7-8e4a2c7f6277 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:55.051660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d963b-0556-4520-8cb7-8e4a2c7f6277 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:55.051932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d963b-0556-4520-8cb7-8e4a2c7f6277 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:55.059956 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d963b-0556-4520-8cb7-8e4a2c7f6277 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:55.061526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d963b-0556-4520-8cb7-8e4a2c7f6277 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:55.071374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 709/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:56.220464 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097213e-c1d8-4aec-9311-c7f9d797eb0b None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:56.221040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097213e-c1d8-4aec-9311-c7f9d797eb0b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:56.221040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097213e-c1d8-4aec-9311-c7f9d797eb0b None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:56.222439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:56.227425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb99fdc-78e7-4059-855d-d7c0992cd61d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:56.227837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb99fdc-78e7-4059-855d-d7c0992cd61d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:56.228023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb99fdc-78e7-4059-855d-d7c0992cd61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:56.246257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-efb99fdc-78e7-4059-855d-d7c0992cd61d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:56.247492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-efb99fdc-78e7-4059-855d-d7c0992cd61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:56.255414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:56.409921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097213e-c1d8-4aec-9311-c7f9d797eb0b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:56.425443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097213e-c1d8-4aec-9311-c7f9d797eb0b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:56.430909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ca3d9-4e2a-4fac-b1ef-2ad2cfbbe6c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:56.431558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ca3d9-4e2a-4fac-b1ef-2ad2cfbbe6c7 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:56.431841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ca3d9-4e2a-4fac-b1ef-2ad2cfbbe6c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:56.432124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ca3d9-4e2a-4fac-b1ef-2ad2cfbbe6c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:56.440362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ca3d9-4e2a-4fac-b1ef-2ad2cfbbe6c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:56.441956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ca3d9-4e2a-4fac-b1ef-2ad2cfbbe6c7 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:56.451395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 727/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:57.536388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcbc61-23af-49c1-b002-c07aa4e061d3 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:57.536926 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcbc61-23af-49c1-b002-c07aa4e061d3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:57.537042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefcbc61-23af-49c1-b002-c07aa4e061d3 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:57.538349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:57.543825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9915b8-3e37-4d14-9d76-9147a6d0531e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:57.544619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9915b8-3e37-4d14-9d76-9147a6d0531e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:57.544619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9915b8-3e37-4d14-9d76-9147a6d0531e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:57.568272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2a9915b8-3e37-4d14-9d76-9147a6d0531e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:57.570366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2a9915b8-3e37-4d14-9d76-9147a6d0531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:57.578744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:57.769979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcbc61-23af-49c1-b002-c07aa4e061d3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:57.785021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefcbc61-23af-49c1-b002-c07aa4e061d3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:16:57.790445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224dfc89-7f1f-4b93-8d85-94e7c88c42c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:16:57.791077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dfc89-7f1f-4b93-8d85-94e7c88c42c2 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:57.791364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dfc89-7f1f-4b93-8d85-94e7c88c42c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:57.791643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dfc89-7f1f-4b93-8d85-94e7c88c42c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:57.800212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224dfc89-7f1f-4b93-8d85-94e7c88c42c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:16:57.801786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224dfc89-7f1f-4b93-8d85-94e7c88c42c2 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:16:57.811235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 712/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:16:59.913543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d4add-1f14-4601-816c-544e6bab00cd None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:59.914105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d4add-1f14-4601-816c-544e6bab00cd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:59.914105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d4add-1f14-4601-816c-544e6bab00cd None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:59.915494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:16:59.920717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5cbf0-b0f4-4e01-a390-4c66de266609 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:16:59.921407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5cbf0-b0f4-4e01-a390-4c66de266609 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:16:59.921538 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5cbf0-b0f4-4e01-a390-4c66de266609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:16:59.939684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e4a5cbf0-b0f4-4e01-a390-4c66de266609 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:16:59.940871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e4a5cbf0-b0f4-4e01-a390-4c66de266609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:16:59.948944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:16:59.988621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d4add-1f14-4601-816c-544e6bab00cd None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:16:59.999140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d4add-1f14-4601-816c-544e6bab00cd None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:00.002847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd37c5-a5fe-4712-885f-4325bdcb2860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:00.003472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd37c5-a5fe-4712-885f-4325bdcb2860 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:00.003808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd37c5-a5fe-4712-885f-4325bdcb2860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:00.004111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd37c5-a5fe-4712-885f-4325bdcb2860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:00.013422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd37c5-a5fe-4712-885f-4325bdcb2860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:00.014943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddd37c5-a5fe-4712-885f-4325bdcb2860 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:00.024983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 730/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:02.438501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980657cc-2f3f-4257-9ac3-933161269eff None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:02.439563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980657cc-2f3f-4257-9ac3-933161269eff None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:02.439677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980657cc-2f3f-4257-9ac3-933161269eff None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:02.440999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:02.446440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603173-f630-4add-b602-8bc7b46b6563 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:02.446912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603173-f630-4add-b602-8bc7b46b6563 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:02.447006 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d603173-f630-4add-b602-8bc7b46b6563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:02.464412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8d603173-f630-4add-b602-8bc7b46b6563 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:02.465598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8d603173-f630-4add-b602-8bc7b46b6563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:02.473567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:02.512021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980657cc-2f3f-4257-9ac3-933161269eff None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:02.523481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980657cc-2f3f-4257-9ac3-933161269eff None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:02.526974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8137e3-b216-43c7-ad84-28ce56201618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:02.527602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8137e3-b216-43c7-ad84-28ce56201618 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:02.527884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8137e3-b216-43c7-ad84-28ce56201618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:02.528763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8137e3-b216-43c7-ad84-28ce56201618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:02.538172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8137e3-b216-43c7-ad84-28ce56201618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:02.539618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8137e3-b216-43c7-ad84-28ce56201618 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:02.550221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 715/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:05.195711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4890a-bb9e-4478-b48a-dc29bef62470 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:05.196324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4890a-bb9e-4478-b48a-dc29bef62470 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:05.196324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4890a-bb9e-4478-b48a-dc29bef62470 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:05.197661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:05.203065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90537bc-3e78-4faf-888f-d61edc9af6b6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:05.203510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90537bc-3e78-4faf-888f-d61edc9af6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:05.203621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90537bc-3e78-4faf-888f-d61edc9af6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:05.221748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e90537bc-3e78-4faf-888f-d61edc9af6b6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:05.223013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e90537bc-3e78-4faf-888f-d61edc9af6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:05.230805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:05.269497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4890a-bb9e-4478-b48a-dc29bef62470 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:05.274931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=55 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:05.280949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4890a-bb9e-4478-b48a-dc29bef62470 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:05.284626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82a6cb-c08b-44ec-b1f8-2863822725f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:05.289430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82a6cb-c08b-44ec-b1f8-2863822725f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:05.289430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82a6cb-c08b-44ec-b1f8-2863822725f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:05.289430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82a6cb-c08b-44ec-b1f8-2863822725f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:05.295771 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82a6cb-c08b-44ec-b1f8-2863822725f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:05.297461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82a6cb-c08b-44ec-b1f8-2863822725f1 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:05.307633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 733/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:07.710117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e722-f817-4a95-b134-8f2982255fd6 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:07.710725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e722-f817-4a95-b134-8f2982255fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:07.710725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228e722-f817-4a95-b134-8f2982255fd6 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:07.712070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:07.717374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82034f-c4ae-4f81-8449-f9d7a3e00222 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:07.717777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82034f-c4ae-4f81-8449-f9d7a3e00222 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:07.717952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82034f-c4ae-4f81-8449-f9d7a3e00222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:07.736197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6b82034f-c4ae-4f81-8449-f9d7a3e00222 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:07.737395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6b82034f-c4ae-4f81-8449-f9d7a3e00222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:07.745253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:07.783612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e722-f817-4a95-b134-8f2982255fd6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:07.794093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228e722-f817-4a95-b134-8f2982255fd6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:07.798401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a61e0-dc02-4e31-842a-96b31f1798df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:07.799137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a61e0-dc02-4e31-842a-96b31f1798df service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:07.799422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a61e0-dc02-4e31-842a-96b31f1798df service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:07.799697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a61e0-dc02-4e31-842a-96b31f1798df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:07.808491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a61e0-dc02-4e31-842a-96b31f1798df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:07.809919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a61e0-dc02-4e31-842a-96b31f1798df service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:07.820048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 718/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:10.216785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75370-e9f6-4833-885b-9cf2b57df582 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:10.217395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75370-e9f6-4833-885b-9cf2b57df582 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:10.217395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75370-e9f6-4833-885b-9cf2b57df582 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:10.218746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:10.224614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd9afc-3997-49ae-b756-a057db01a5b4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:10.225047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd9afc-3997-49ae-b756-a057db01a5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:10.225221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd9afc-3997-49ae-b756-a057db01a5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:10.243309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ccfd9afc-3997-49ae-b756-a057db01a5b4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:10.244502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ccfd9afc-3997-49ae-b756-a057db01a5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:10.252337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:10.289875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e75370-e9f6-4833-885b-9cf2b57df582 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:10.301455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e75370-e9f6-4833-885b-9cf2b57df582 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:10.304953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f464b-e7f4-4cfc-996e-feded651c0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:10.305592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f464b-e7f4-4cfc-996e-feded651c0ba service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:10.305880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f464b-e7f4-4cfc-996e-feded651c0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:10.306159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f464b-e7f4-4cfc-996e-feded651c0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:10.315179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f464b-e7f4-4cfc-996e-feded651c0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:10.316613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f464b-e7f4-4cfc-996e-feded651c0ba service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:10.326136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 736/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:12.707222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b5814-bc19-4bce-bcdb-9bde2ddf6142 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:12.707807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b5814-bc19-4bce-bcdb-9bde2ddf6142 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:12.707807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b5814-bc19-4bce-bcdb-9bde2ddf6142 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:12.709111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:12.714888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad582e14-f396-4f15-9598-5e1d250e2b29 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:12.715384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad582e14-f396-4f15-9598-5e1d250e2b29 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:12.715548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad582e14-f396-4f15-9598-5e1d250e2b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:12.736961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ad582e14-f396-4f15-9598-5e1d250e2b29 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:12.738173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ad582e14-f396-4f15-9598-5e1d250e2b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:12.746287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:12.786138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b5814-bc19-4bce-bcdb-9bde2ddf6142 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:12.798659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b5814-bc19-4bce-bcdb-9bde2ddf6142 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:12.804329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c1449-7cf2-405f-92c4-1090c937f9a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:12.804329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c1449-7cf2-405f-92c4-1090c937f9a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:12.804329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c1449-7cf2-405f-92c4-1090c937f9a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:12.804329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c1449-7cf2-405f-92c4-1090c937f9a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:12.813931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c1449-7cf2-405f-92c4-1090c937f9a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:12.816179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c1449-7cf2-405f-92c4-1090c937f9a8 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:12.827858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 721/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:15.279276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6522e4-5769-41c2-ac41-1a6c5266c35a None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:15.279844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6522e4-5769-41c2-ac41-1a6c5266c35a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:15.279844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6522e4-5769-41c2-ac41-1a6c5266c35a None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:15.281116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:15.287418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8ab1-d92b-4b0c-b9ad-191f6d784c7d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:15.287933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8ab1-d92b-4b0c-b9ad-191f6d784c7d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:15.288119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb8ab1-d92b-4b0c-b9ad-191f6d784c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:15.294905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=70 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:15.308435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b3bb8ab1-d92b-4b0c-b9ad-191f6d784c7d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:15.314437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b3bb8ab1-d92b-4b0c-b9ad-191f6d784c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:15.319062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:15.368993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6522e4-5769-41c2-ac41-1a6c5266c35a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:15.386791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6522e4-5769-41c2-ac41-1a6c5266c35a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:15.392402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d008f2-eda9-437a-82ae-cb183ff9a12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:15.393061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d008f2-eda9-437a-82ae-cb183ff9a12b service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:15.393353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d008f2-eda9-437a-82ae-cb183ff9a12b service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:15.393639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d008f2-eda9-437a-82ae-cb183ff9a12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:15.403250 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d008f2-eda9-437a-82ae-cb183ff9a12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:15.404687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d008f2-eda9-437a-82ae-cb183ff9a12b service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:15.414734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 739/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:17.830579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a939851-4c2e-4829-b0c0-3aafc78445b4 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:17.831103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a939851-4c2e-4829-b0c0-3aafc78445b4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:17.831103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a939851-4c2e-4829-b0c0-3aafc78445b4 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:17.832456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:17.838314 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f265c2f-f12e-4e9a-b08f-fa8537b86179 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:17.838979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f265c2f-f12e-4e9a-b08f-fa8537b86179 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:17.838979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f265c2f-f12e-4e9a-b08f-fa8537b86179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:17.860598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9f265c2f-f12e-4e9a-b08f-fa8537b86179 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:17.861918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9f265c2f-f12e-4e9a-b08f-fa8537b86179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:17.869828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:17.907754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a939851-4c2e-4829-b0c0-3aafc78445b4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:17.919368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a939851-4c2e-4829-b0c0-3aafc78445b4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:17.923637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951edb3-3768-47bf-b746-6af6f611a1b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:17.924288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951edb3-3768-47bf-b746-6af6f611a1b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:17.924568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951edb3-3768-47bf-b746-6af6f611a1b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:17.924843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951edb3-3768-47bf-b746-6af6f611a1b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:17.937055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951edb3-3768-47bf-b746-6af6f611a1b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:17.942019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951edb3-3768-47bf-b746-6af6f611a1b4 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:17.957222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 724/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:20.297435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792a58c-a051-45fe-83f5-cabe1cd47866 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:20.297976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792a58c-a051-45fe-83f5-cabe1cd47866 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:20.297976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792a58c-a051-45fe-83f5-cabe1cd47866 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:20.299357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:20.304765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7274752-0c20-47a3-b378-141f25e0642e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:20.305197 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7274752-0c20-47a3-b378-141f25e0642e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:20.305314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7274752-0c20-47a3-b378-141f25e0642e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:20.323249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a7274752-0c20-47a3-b378-141f25e0642e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:20.324429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a7274752-0c20-47a3-b378-141f25e0642e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:20.332854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:20.368614 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792a58c-a051-45fe-83f5-cabe1cd47866 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:20.379571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792a58c-a051-45fe-83f5-cabe1cd47866 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:20.383068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852b21a-43e4-4d63-827d-d730733631f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:20.383692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852b21a-43e4-4d63-827d-d730733631f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:20.383976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852b21a-43e4-4d63-827d-d730733631f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:20.384254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852b21a-43e4-4d63-827d-d730733631f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:20.393950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852b21a-43e4-4d63-827d-d730733631f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:20.395414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852b21a-43e4-4d63-827d-d730733631f7 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:20.405122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 742/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:22.676062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11990aef-5e8a-43b8-8925-49338aef3ad0 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:22.676603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11990aef-5e8a-43b8-8925-49338aef3ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:22.676603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11990aef-5e8a-43b8-8925-49338aef3ad0 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:22.678018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:22.683435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50049e4-75d6-4af2-9f07-7d7e1e486e8e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:22.683832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50049e4-75d6-4af2-9f07-7d7e1e486e8e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:22.683991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50049e4-75d6-4af2-9f07-7d7e1e486e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:22.701955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f50049e4-75d6-4af2-9f07-7d7e1e486e8e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:22.703209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f50049e4-75d6-4af2-9f07-7d7e1e486e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:22.710896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:22.748172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11990aef-5e8a-43b8-8925-49338aef3ad0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:22.758868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11990aef-5e8a-43b8-8925-49338aef3ad0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:22.762324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c590a8-70df-416d-8c24-ccb8814e3aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:22.763011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c590a8-70df-416d-8c24-ccb8814e3aed service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:22.763302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c590a8-70df-416d-8c24-ccb8814e3aed service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:22.763575 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c590a8-70df-416d-8c24-ccb8814e3aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:22.773294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c590a8-70df-416d-8c24-ccb8814e3aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:22.774760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c590a8-70df-416d-8c24-ccb8814e3aed service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:22.784701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 727/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:32.698428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=35 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:32.779950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=17 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:17:56.548829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb5e3b-7b60-47c4-ac19-ee2b6e7121fb None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.549472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb5e3b-7b60-47c4-ac19-ee2b6e7121fb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.549472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb5e3b-7b60-47c4-ac19-ee2b6e7121fb None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.550800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 744/1471] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:56.556421 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1432a-e966-449d-bb9d-3a629402b3a5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.557263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1432a-e966-449d-bb9d-3a629402b3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.557436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1432a-e966-449d-bb9d-3a629402b3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.576187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-30f1432a-e966-449d-bb9d-3a629402b3a5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:56.578212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-30f1432a-e966-449d-bb9d-3a629402b3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:56.586168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 728/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:56.632966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680fdfb-e046-4bed-b79b-37fad652948f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.633255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680fdfb-e046-4bed-b79b-37fad652948f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.633536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680fdfb-e046-4bed-b79b-37fad652948f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.634931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 745/1473] 10.222.0.57 () {54 vars in 962 bytes} [Thu May 19 12:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:56.640446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2340c4-7031-4fcf-a282-0aac57f52c85 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.640737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2340c4-7031-4fcf-a282-0aac57f52c85 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.641022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2340c4-7031-4fcf-a282-0aac57f52c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.659133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9a2340c4-7031-4fcf-a282-0aac57f52c85 None None] MFA Rules not processed for user `9969632e796345fa9671a45cf61ede3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:56.660313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9a2340c4-7031-4fcf-a282-0aac57f52c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:56.666580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 729/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Thu May 19 12:17:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:56.674403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680fdfb-e046-4bed-b79b-37fad652948f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:56.684953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680fdfb-e046-4bed-b79b-37fad652948f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:56.689023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a4645-8390-47ae-972c-28f19eaaeed6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9969632e796345fa9671a45cf61ede3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:56.689653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a4645-8390-47ae-972c-28f19eaaeed6 service glance] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:56.689939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a4645-8390-47ae-972c-28f19eaaeed6 service glance] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:56.690218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a4645-8390-47ae-972c-28f19eaaeed6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:56.698973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a4645-8390-47ae-972c-28f19eaaeed6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:56.700415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a4645-8390-47ae-972c-28f19eaaeed6 service glance] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:56.710293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 746/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:17:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:17:59.165987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52743cff-c466-45b7-9430-47e94a6443fa None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:59.166591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52743cff-c466-45b7-9430-47e94a6443fa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:59.166591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52743cff-c466-45b7-9430-47e94a6443fa None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:59.167930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 730/1476] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:17:59.174343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89978b3b-33a0-4e45-aa93-0b2341356bb0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:59.174840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89978b3b-33a0-4e45-aa93-0b2341356bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:59.175007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89978b3b-33a0-4e45-aa93-0b2341356bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:59.192128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-89978b3b-33a0-4e45-aa93-0b2341356bb0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:17:59.193305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-89978b3b-33a0-4e45-aa93-0b2341356bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:17:59.201556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 747/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:17:59.225003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52743cff-c466-45b7-9430-47e94a6443fa None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:17:59.238198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52743cff-c466-45b7-9430-47e94a6443fa None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:17:59.242764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa396dde-0012-4706-9a72-5be9a7cd5f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:17:59.243410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa396dde-0012-4706-9a72-5be9a7cd5f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:17:59.243691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa396dde-0012-4706-9a72-5be9a7cd5f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:17:59.243966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa396dde-0012-4706-9a72-5be9a7cd5f52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:17:59.244702 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa396dde-0012-4706-9a72-5be9a7cd5f52 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:17:59.246444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-aa396dde-0012-4706-9a72-5be9a7cd5f52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:17:59.246444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 19 12:17:59.246444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: May 19 12:17:59.247598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa396dde-0012-4706-9a72-5be9a7cd5f52 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:17:59.267482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 731/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 19 12:17:59 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:18:01.584333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c46cb5-d361-4845-bb21-6e1e1f030a17 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:01.584908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c46cb5-d361-4845-bb21-6e1e1f030a17 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:01.584908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c46cb5-d361-4845-bb21-6e1e1f030a17 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:01.586269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 748/1479] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:18:01.591661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45677a69-abd0-480a-b45e-8660b702405e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:01.592204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45677a69-abd0-480a-b45e-8660b702405e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:01.592204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45677a69-abd0-480a-b45e-8660b702405e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:01.611111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-45677a69-abd0-480a-b45e-8660b702405e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:01.612317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-45677a69-abd0-480a-b45e-8660b702405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:01.620313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 732/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:01.663110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c46cb5-d361-4845-bb21-6e1e1f030a17 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:01.678384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c46cb5-d361-4845-bb21-6e1e1f030a17 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:01.684117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d7ca1-266c-4200-80ab-c605060e2589 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:01.684757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d7ca1-266c-4200-80ab-c605060e2589 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:01.685040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d7ca1-266c-4200-80ab-c605060e2589 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:01.685923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d7ca1-266c-4200-80ab-c605060e2589 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:01.694148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d7ca1-266c-4200-80ab-c605060e2589 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:01.695893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d7ca1-266c-4200-80ab-c605060e2589 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:01.705291 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 749/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:18:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:18:04.202343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e50a40-5491-4e6e-8a40-ce6002e0e5ad None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:04.202928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e50a40-5491-4e6e-8a40-ce6002e0e5ad None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:04.202928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e50a40-5491-4e6e-8a40-ce6002e0e5ad None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:04.204267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 733/1482] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:18:04.209432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381423fa-f6b4-4ffc-a319-35a75db6eb95 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:04.209886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381423fa-f6b4-4ffc-a319-35a75db6eb95 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:04.209984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381423fa-f6b4-4ffc-a319-35a75db6eb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:04.227978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-381423fa-f6b4-4ffc-a319-35a75db6eb95 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:04.229200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-381423fa-f6b4-4ffc-a319-35a75db6eb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:04.237586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 750/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:04.272537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e50a40-5491-4e6e-8a40-ce6002e0e5ad None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:04.284508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e50a40-5491-4e6e-8a40-ce6002e0e5ad None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:04.287883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a76eba-b106-42c3-97b3-5ab84e3348a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:04.288515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a76eba-b106-42c3-97b3-5ab84e3348a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:04.288795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a76eba-b106-42c3-97b3-5ab84e3348a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:04.289067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a76eba-b106-42c3-97b3-5ab84e3348a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:04.298752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a76eba-b106-42c3-97b3-5ab84e3348a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:04.300153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a76eba-b106-42c3-97b3-5ab84e3348a4 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:04.309345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 734/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:06.635495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd69e-3d30-4623-bee6-4b5be004d579 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.636240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd69e-3d30-4623-bee6-4b5be004d579 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.636240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd69e-3d30-4623-bee6-4b5be004d579 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.637415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 751/1485] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:18:06.642882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d64b1b-b8e9-49b9-b0e8-7cfa30225d2e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.643362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d64b1b-b8e9-49b9-b0e8-7cfa30225d2e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.643466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d64b1b-b8e9-49b9-b0e8-7cfa30225d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.663833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b3d64b1b-b8e9-49b9-b0e8-7cfa30225d2e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:06.665290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b3d64b1b-b8e9-49b9-b0e8-7cfa30225d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:06.675252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 735/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:06.714047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961dd69e-3d30-4623-bee6-4b5be004d579 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:06.715830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-961dd69e-3d30-4623-bee6-4b5be004d579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:06.729317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961dd69e-3d30-4623-bee6-4b5be004d579 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:06.734118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c7ef4-5f71-482b-8761-4fd88330f0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:06.734803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7ef4-5f71-482b-8761-4fd88330f0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:06.735126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7ef4-5f71-482b-8761-4fd88330f0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:06.735387 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c7ef4-5f71-482b-8761-4fd88330f0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:06.746274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c7ef4-5f71-482b-8761-4fd88330f0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:06.754634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c7ef4-5f71-482b-8761-4fd88330f0d2 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:06.767309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 752/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:09.280452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02904484-944c-450d-8b34-e153340b132e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:09.280999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02904484-944c-450d-8b34-e153340b132e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:09.280999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02904484-944c-450d-8b34-e153340b132e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:09.282295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 736/1488] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:18:09.287818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffdcd3-c1ca-4234-bf67-ab2e7056e370 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:09.288323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffdcd3-c1ca-4234-bf67-ab2e7056e370 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:09.288435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffdcd3-c1ca-4234-bf67-ab2e7056e370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:09.306792 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e4ffdcd3-c1ca-4234-bf67-ab2e7056e370 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:09.307990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e4ffdcd3-c1ca-4234-bf67-ab2e7056e370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:09.316547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 753/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:09.357173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02904484-944c-450d-8b34-e153340b132e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:09.369161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02904484-944c-450d-8b34-e153340b132e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:09.373367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab8d42-a5ab-4b61-9184-ae404f3907c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:09.374003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab8d42-a5ab-4b61-9184-ae404f3907c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:09.374319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab8d42-a5ab-4b61-9184-ae404f3907c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:09.374632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab8d42-a5ab-4b61-9184-ae404f3907c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:09.383661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab8d42-a5ab-4b61-9184-ae404f3907c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:09.385179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab8d42-a5ab-4b61-9184-ae404f3907c2 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:09.395808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 737/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:12.084690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653bd56-02ca-482c-a594-3d8e22db058c None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:12.085711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653bd56-02ca-482c-a594-3d8e22db058c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:12.085711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653bd56-02ca-482c-a594-3d8e22db058c None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:12.086688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 754/1491] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 19 12:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:18:12.092120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b7a74-c925-4e2e-b236-268cf45a7bb0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:12.092719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b7a74-c925-4e2e-b236-268cf45a7bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:12.092719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b7a74-c925-4e2e-b236-268cf45a7bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:12.111851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-147b7a74-c925-4e2e-b236-268cf45a7bb0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:18:12.114157 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-147b7a74-c925-4e2e-b236-268cf45a7bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:18:12.121561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 738/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 19 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:18:12.161207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653bd56-02ca-482c-a594-3d8e22db058c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:18:12.174120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653bd56-02ca-482c-a594-3d8e22db058c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:18:12.178092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629b717-f1ed-4114-8a4e-feab78fff267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:18:12.178810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629b717-f1ed-4114-8a4e-feab78fff267 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:18:12.179094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629b717-f1ed-4114-8a4e-feab78fff267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:18:12.179378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629b717-f1ed-4114-8a4e-feab78fff267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:18:12.189535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629b717-f1ed-4114-8a4e-feab78fff267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:18:12.191041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629b717-f1ed-4114-8a4e-feab78fff267 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:18:12.200701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 755/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:18:22.105208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=33 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:18:22.196335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=33 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:06.267567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc1fe7-2259-49e5-8bd5-b04c27dc9bb4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:06.268165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc1fe7-2259-49e5-8bd5-b04c27dc9bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:06.268272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc1fe7-2259-49e5-8bd5-b04c27dc9bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:06.292377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fdfc1fe7-2259-49e5-8bd5-b04c27dc9bb4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:06.293490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fdfc1fe7-2259-49e5-8bd5-b04c27dc9bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:06.301606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 739/1494] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:06.311991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbdb79-7db4-4876-b3be-c4bb81399e96 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:06.312593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbdb79-7db4-4876-b3be-c4bb81399e96 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:06.312593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbdb79-7db4-4876-b3be-c4bb81399e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:06.331952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-89dbdb79-7db4-4876-b3be-c4bb81399e96 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:06.333160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-89dbdb79-7db4-4876-b3be-c4bb81399e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:06.341042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 756/1495] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:06.345782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc1fe7-2259-49e5-8bd5-b04c27dc9bb4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:06.355519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc1fe7-2259-49e5-8bd5-b04c27dc9bb4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:06.360009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:06.360659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:06.360943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:06.361223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:06.361949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:20:06.362265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:06.363311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:20:06.363311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 19 12:20:06.363311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: May 19 12:20:06.364208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:06.376497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 740/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:20:06.385564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbdb79-7db4-4876-b3be-c4bb81399e96 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:06.395002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbdb79-7db4-4876-b3be-c4bb81399e96 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:06.399749 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da2244-0e6d-4989-a5b5-c00aaa0afb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:06.400405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da2244-0e6d-4989-a5b5-c00aaa0afb5c admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:06.400693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da2244-0e6d-4989-a5b5-c00aaa0afb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:06.400970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da2244-0e6d-4989-a5b5-c00aaa0afb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:06.401672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da2244-0e6d-4989-a5b5-c00aaa0afb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:06.403448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-49da2244-0e6d-4989-a5b5-c00aaa0afb5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:20:06.403448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 19 12:20:06.403448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: May 19 12:20:06.404627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da2244-0e6d-4989-a5b5-c00aaa0afb5c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:06.582847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 757/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:20:06 2022] POST /identity/v3/projects => generated 379 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:20:06.588263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:06.599283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bbefb-59bd-494e-afdb-a2d9f923914b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:06.665967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692218f-0786-416d-8490-410166858b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:06.666793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692218f-0786-416d-8490-410166858b33 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:06.667100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692218f-0786-416d-8490-410166858b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:06.667357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692218f-0786-416d-8490-410166858b33 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:06.668063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692218f-0786-416d-8490-410166858b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:06.669073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-6692218f-0786-416d-8490-410166858b33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:20:06.669073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 19 12:20:06.669073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: May 19 12:20:06.670581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692218f-0786-416d-8490-410166858b33 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:06.777816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 741/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:20:06 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:20:06.782105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da2244-0e6d-4989-a5b5-c00aaa0afb5c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:06.792421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da2244-0e6d-4989-a5b5-c00aaa0afb5c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:06.797238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783e9e2-2f07-45de-89fa-9650997d1b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:06.797900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783e9e2-2f07-45de-89fa-9650997d1b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:06.798205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783e9e2-2f07-45de-89fa-9650997d1b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:06.798521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783e9e2-2f07-45de-89fa-9650997d1b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:06.799252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783e9e2-2f07-45de-89fa-9650997d1b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:06.800264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-4783e9e2-2f07-45de-89fa-9650997d1b0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:20:06.800264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 19 12:20:06.800264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: May 19 12:20:06.801143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783e9e2-2f07-45de-89fa-9650997d1b0f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:06.814660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 758/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:06.818898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692218f-0786-416d-8490-410166858b33 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:06.829370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692218f-0786-416d-8490-410166858b33 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:06.833977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e101fe-ac04-4abd-9523-25ddbd2d1663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:06.834620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e101fe-ac04-4abd-9523-25ddbd2d1663 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:06.835493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e101fe-ac04-4abd-9523-25ddbd2d1663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:06.835772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e101fe-ac04-4abd-9523-25ddbd2d1663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:06.836512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e101fe-ac04-4abd-9523-25ddbd2d1663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:06.837516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-c8e101fe-ac04-4abd-9523-25ddbd2d1663 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:20:06.837516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 19 12:20:06.837516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: May 19 12:20:06.838430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e101fe-ac04-4abd-9523-25ddbd2d1663 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:06.852067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 742/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:20:06.856239 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783e9e2-2f07-45de-89fa-9650997d1b0f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:06.866231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783e9e2-2f07-45de-89fa-9650997d1b0f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:06.870846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54c1e4-7c01-4eb2-b7f1-ed09f24eb3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:06.871728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c1e4-7c01-4eb2-b7f1-ed09f24eb3f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:06.872017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c1e4-7c01-4eb2-b7f1-ed09f24eb3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:06.872300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c1e4-7c01-4eb2-b7f1-ed09f24eb3f0 admin admin] PATH_INFO: `/v3/projects/773f5dde778a471b8b64b5af5922a9a4/users/cfc7a396ea96429a8de9a4f7ae26669a/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:06.887378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54c1e4-7c01-4eb2-b7f1-ed09f24eb3f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=773f5dde778a471b8b64b5af5922a9a4, user_id=cfc7a396ea96429a8de9a4f7ae26669a, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:06.888419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-6e54c1e4-7c01-4eb2-b7f1-ed09f24eb3f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:20:06.888419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 19 12:20:06.888419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: May 19 12:20:06.890348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54c1e4-7c01-4eb2-b7f1-ed09f24eb3f0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:07.012262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 759/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:20:06 2022] PUT /identity/v3/projects/773f5dde778a471b8b64b5af5922a9a4/users/cfc7a396ea96429a8de9a4f7ae26669a/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:07.024502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfa05-f3ce-4853-8064-e93f3e65cd07 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:07.024787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfa05-f3ce-4853-8064-e93f3e65cd07 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:07.025147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347cfa05-f3ce-4853-8064-e93f3e65cd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:07.052566 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-347cfa05-f3ce-4853-8064-e93f3e65cd07 None None] MFA Rules not processed for user `cfc7a396ea96429a8de9a4f7ae26669a`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:07.054468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-347cfa05-f3ce-4853-8064-e93f3e65cd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:07.119736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 743/1502] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:20:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:07.133535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14767570-cd54-4df0-b69f-468445002d8f None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:07.133831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14767570-cd54-4df0-b69f-468445002d8f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:07.134137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14767570-cd54-4df0-b69f-468445002d8f None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:07.135591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 760/1503] 10.222.0.57 () {54 vars in 963 bytes} [Thu May 19 12:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:20:07.140519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732589a2-e336-4818-b3e0-ba9a6119cecb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:07.140816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732589a2-e336-4818-b3e0-ba9a6119cecb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:07.141100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732589a2-e336-4818-b3e0-ba9a6119cecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:07.159740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-732589a2-e336-4818-b3e0-ba9a6119cecb None None] MFA Rules not processed for user `316ad1c2eccf4642a21c68856585ceee`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:07.161016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-732589a2-e336-4818-b3e0-ba9a6119cecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:07.212197 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:07.219865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 744/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Thu May 19 12:20:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:20:07.228402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14767570-cd54-4df0-b69f-468445002d8f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:07.239837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14767570-cd54-4df0-b69f-468445002d8f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:07.243884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d055a4-98ab-4a43-a440-30a2505baf94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:07.244534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d055a4-98ab-4a43-a440-30a2505baf94 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:07.244823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d055a4-98ab-4a43-a440-30a2505baf94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:07.245142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d055a4-98ab-4a43-a440-30a2505baf94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:07.258178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d055a4-98ab-4a43-a440-30a2505baf94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:07.259675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d055a4-98ab-4a43-a440-30a2505baf94 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:07.268991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 761/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:20:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:20:07.318461 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8facd-b128-47a4-8b07-ab59d3a698ac None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:07.318899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8facd-b128-47a4-8b07-ab59d3a698ac None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:07.319018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8facd-b128-47a4-8b07-ab59d3a698ac None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:07.320346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 745/1506] 10.222.0.57 () {52 vars in 829 bytes} [Thu May 19 12:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:20:07.323996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d055a4-98ab-4a43-a440-30a2505baf94 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:07.398058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d055a4-98ab-4a43-a440-30a2505baf94 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:07.403034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:07.403932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:07.404212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:07.404487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] PATH_INFO: `/v3/users/cfc7a396ea96429a8de9a4f7ae26669a` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:07.405725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc7a396ea96429a8de9a4f7ae26669a)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:07.406807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:20:07.406807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 19 12:20:07.406807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: May 19 12:20:07.408021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:07.925597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc7a396ea96429a8de9a4f7ae26669a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:07.940642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc7a396ea96429a8de9a4f7ae26669a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:07.941514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc7a396ea96429a8de9a4f7ae26669a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:07.942137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc7a396ea96429a8de9a4f7ae26669a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:08.057164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc7a396ea96429a8de9a4f7ae26669a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:08.077396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:08.087282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 762/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:20:07 2022] DELETE /identity/v3/users/cfc7a396ea96429a8de9a4f7ae26669a => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:08.091894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07d5de-7763-4460-814c-3cbf929d171c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:08.092188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07d5de-7763-4460-814c-3cbf929d171c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:08.092473 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07d5de-7763-4460-814c-3cbf929d171c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:08.112565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fd07d5de-7763-4460-814c-3cbf929d171c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:20:08.113887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fd07d5de-7763-4460-814c-3cbf929d171c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:08.183975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 746/1508] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:20:08.195046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:08.198910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:08.284925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e4ae1-700d-4cd1-8eff-6694a8288622 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:08.290919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c433fa-920c-439b-b707-f4c94159b95b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:08.292048 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c433fa-920c-439b-b707-f4c94159b95b service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:08.292540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c433fa-920c-439b-b707-f4c94159b95b service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:08.293031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c433fa-920c-439b-b707-f4c94159b95b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:08.310213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c433fa-920c-439b-b707-f4c94159b95b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:08.312949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c433fa-920c-439b-b707-f4c94159b95b service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:08.329999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 763/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:20:09.274012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07d5de-7763-4460-814c-3cbf929d171c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:20:09.276500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fd07d5de-7763-4460-814c-3cbf929d171c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:20:09.289984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07d5de-7763-4460-814c-3cbf929d171c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:20:09.295615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:20:09.296510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:20:09.296772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:20:09.297191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] PATH_INFO: `/v3/projects/773f5dde778a471b8b64b5af5922a9a4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:20:09.298440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=773f5dde778a471b8b64b5af5922a9a4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:20:09.299705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:20:09.299705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 19 12:20:09.299705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: May 19 12:20:09.301023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:20:09.483621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773f5dde778a471b8b64b5af5922a9a4', 'request_id': 'req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:09.495592 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773f5dde778a471b8b64b5af5922a9a4', 'request_id': 'req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:09.590642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773f5dde778a471b8b64b5af5922a9a4', 'request_id': 'req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:09.608981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] The token cache is being invalidate because project 773f5dde778a471b8b64b5af5922a9a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:20:09.609346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:09.610241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-79cc3eab-2222-45fe-ab74-a0094a9b3a4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:20:09.612766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 747/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:20:09 2022] DELETE /identity/v3/projects/773f5dde778a471b8b64b5af5922a9a4 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:20:18.326773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:20:19.605272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:22:01.318950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d883c-d442-46c7-bed9-24989b5890f1 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:01.319521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d883c-d442-46c7-bed9-24989b5890f1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:01.319904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d883c-d442-46c7-bed9-24989b5890f1 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:01.322673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:22:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:22:01.329458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6839d1-9952-4550-b447-f2ec8acf51e7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:01.330201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6839d1-9952-4550-b447-f2ec8acf51e7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:01.330201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6839d1-9952-4550-b447-f2ec8acf51e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:01.366292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ab6839d1-9952-4550-b447-f2ec8acf51e7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:01.368254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ab6839d1-9952-4550-b447-f2ec8acf51e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:01.447020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:22:01.635399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d883c-d442-46c7-bed9-24989b5890f1 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:01.637203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d41d883c-d442-46c7-bed9-24989b5890f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:01.731341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d883c-d442-46c7-bed9-24989b5890f1 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:01.737835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acea0f-a6ec-4de5-8f94-b43b95baa3cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:01.738548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acea0f-a6ec-4de5-8f94-b43b95baa3cb service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:01.738836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acea0f-a6ec-4de5-8f94-b43b95baa3cb service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:01.739119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acea0f-a6ec-4de5-8f94-b43b95baa3cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:01.748980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acea0f-a6ec-4de5-8f94-b43b95baa3cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:01.751404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acea0f-a6ec-4de5-8f94-b43b95baa3cb service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:01.763080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 765/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:02.871796 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849e879-e70e-411a-bb26-469f713160c6 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:02.872504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849e879-e70e-411a-bb26-469f713160c6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:02.872788 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849e879-e70e-411a-bb26-469f713160c6 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:02.875545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:22:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:22:02.884028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacdfef-e24b-4768-9ff3-b65a3fdabedc None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:02.884779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacdfef-e24b-4768-9ff3-b65a3fdabedc None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:02.885064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacdfef-e24b-4768-9ff3-b65a3fdabedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:02.914428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-caacdfef-e24b-4768-9ff3-b65a3fdabedc None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:02.915784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-caacdfef-e24b-4768-9ff3-b65a3fdabedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:02.925723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:03.106898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849e879-e70e-411a-bb26-469f713160c6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:03.133580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849e879-e70e-411a-bb26-469f713160c6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:03.143972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae22ee-2f4d-49a0-afab-15357505705b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:03.145086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae22ee-2f4d-49a0-afab-15357505705b service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:03.145584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae22ee-2f4d-49a0-afab-15357505705b service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:03.146081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae22ee-2f4d-49a0-afab-15357505705b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:03.161319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae22ee-2f4d-49a0-afab-15357505705b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:03.163080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae22ee-2f4d-49a0-afab-15357505705b service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:03.182540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 750/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:03.911895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46e3c6-e8e9-4f0a-9916-f06027ff88d4 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:03.912788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46e3c6-e8e9-4f0a-9916-f06027ff88d4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:03.912967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46e3c6-e8e9-4f0a-9916-f06027ff88d4 None None] PATH_INFO: `/v3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:03.914486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 767/1517] 10.222.0.77 () {54 vars in 925 bytes} [Thu May 19 12:22:03 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:22:03.925368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c889b5-3546-4cd2-bd2b-5160116d6ce2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:03.925961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c889b5-3546-4cd2-bd2b-5160116d6ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:03.925961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c889b5-3546-4cd2-bd2b-5160116d6ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:03.956217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-68c889b5-3546-4cd2-bd2b-5160116d6ce2 None None] MFA Rules not processed for user `4f41920bad3147bba0000de911dfe5a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:03.958246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-68c889b5-3546-4cd2-bd2b-5160116d6ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:04.090798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 751/1518] 10.222.0.77 () {58 vars in 1015 bytes} [Thu May 19 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:08.617820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6679-6856-498a-a7ba-b954e6aa60c8 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:08.618551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6679-6856-498a-a7ba-b954e6aa60c8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:08.618551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6679-6856-498a-a7ba-b954e6aa60c8 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:08.620097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 768/1519] 10.222.0.57 () {54 vars in 960 bytes} [Thu May 19 12:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:22:08.628051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024264d-8b53-415f-b239-f3831e8590b7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:08.628640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024264d-8b53-415f-b239-f3831e8590b7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:08.628750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024264d-8b53-415f-b239-f3831e8590b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:08.650187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7024264d-8b53-415f-b239-f3831e8590b7 None None] MFA Rules not processed for user `4f41920bad3147bba0000de911dfe5a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:08.651454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7024264d-8b53-415f-b239-f3831e8590b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:08.660957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 752/1520] 10.222.0.57 () {58 vars in 1057 bytes} [Thu May 19 12:22:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:08.668906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6679-6856-498a-a7ba-b954e6aa60c8 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:08.683322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6679-6856-498a-a7ba-b954e6aa60c8 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:08.690076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217368cc-1985-4c1b-b172-3fac59f9037d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:08.690823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217368cc-1985-4c1b-b172-3fac59f9037d service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:08.691107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217368cc-1985-4c1b-b172-3fac59f9037d service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:08.691381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217368cc-1985-4c1b-b172-3fac59f9037d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:08.700501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217368cc-1985-4c1b-b172-3fac59f9037d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:08.702101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217368cc-1985-4c1b-b172-3fac59f9037d service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:08.716970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 769/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:22:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:14.709035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b056-e285-4676-81dc-047f19b242bc None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:14.709674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b056-e285-4676-81dc-047f19b242bc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:14.709674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b056-e285-4676-81dc-047f19b242bc None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:14.711064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:22:14.717905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf6abf-afc2-483e-8670-91b664bf12ce None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:14.718528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf6abf-afc2-483e-8670-91b664bf12ce None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:14.719659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf6abf-afc2-483e-8670-91b664bf12ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:14.750725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-32cf6abf-afc2-483e-8670-91b664bf12ce None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:14.752921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-32cf6abf-afc2-483e-8670-91b664bf12ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:14.767759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:14.958497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b056-e285-4676-81dc-047f19b242bc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:14.975048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b056-e285-4676-81dc-047f19b242bc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:14.980768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67f69c-f71f-4c01-9114-fa8616507b1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:14.985257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67f69c-f71f-4c01-9114-fa8616507b1f service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:14.985257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67f69c-f71f-4c01-9114-fa8616507b1f service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:14.985257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67f69c-f71f-4c01-9114-fa8616507b1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:14.994965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67f69c-f71f-4c01-9114-fa8616507b1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:14.996741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67f69c-f71f-4c01-9114-fa8616507b1f service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:15.008203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 754/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:22:16.005577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bff46a-ad32-458e-9004-48f1b332177c None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:16.006288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bff46a-ad32-458e-9004-48f1b332177c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:16.006581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bff46a-ad32-458e-9004-48f1b332177c None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:16.009103 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 12:22:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:22:16.014987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae10ab9-f80b-4531-8107-94ae4a2e6747 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:16.015902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae10ab9-f80b-4531-8107-94ae4a2e6747 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:16.015902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae10ab9-f80b-4531-8107-94ae4a2e6747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:16.039327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5ae10ab9-f80b-4531-8107-94ae4a2e6747 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:16.040591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5ae10ab9-f80b-4531-8107-94ae4a2e6747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:16.049945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:16.208559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bff46a-ad32-458e-9004-48f1b332177c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:16.236714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bff46a-ad32-458e-9004-48f1b332177c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:16.245986 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a1843-e3f6-497b-9838-ae9676ebe6c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:16.247008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a1843-e3f6-497b-9838-ae9676ebe6c9 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:16.247519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a1843-e3f6-497b-9838-ae9676ebe6c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:16.248023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a1843-e3f6-497b-9838-ae9676ebe6c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:16.265701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a1843-e3f6-497b-9838-ae9676ebe6c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:16.269335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a1843-e3f6-497b-9838-ae9676ebe6c9 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:16.289092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 772/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:20.681373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e2c85-2742-4b81-85c2-8e86e0ea193a None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:20.681946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e2c85-2742-4b81-85c2-8e86e0ea193a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:20.681946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e2c85-2742-4b81-85c2-8e86e0ea193a None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:20.683417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:22:20.689826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d53815-0151-43ea-9421-872e346209f5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:20.690318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d53815-0151-43ea-9421-872e346209f5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:20.690451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d53815-0151-43ea-9421-872e346209f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:20.712664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-42d53815-0151-43ea-9421-872e346209f5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:20.714038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-42d53815-0151-43ea-9421-872e346209f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:20.723351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:20.760725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e2c85-2742-4b81-85c2-8e86e0ea193a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:20.763394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-440e2c85-2742-4b81-85c2-8e86e0ea193a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:20.830244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e2c85-2742-4b81-85c2-8e86e0ea193a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:20.833916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6a476-2a4f-49aa-9d1a-decb58e9c9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:20.834613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6a476-2a4f-49aa-9d1a-decb58e9c9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:20.834930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6a476-2a4f-49aa-9d1a-decb58e9c9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:20.835214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6a476-2a4f-49aa-9d1a-decb58e9c9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:20.845654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6a476-2a4f-49aa-9d1a-decb58e9c9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:20.847151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6a476-2a4f-49aa-9d1a-decb58e9c9d3 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:20.857695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:22:20.857952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 757/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:22:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:23.164869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d63b-8b88-4964-8ee5-369d1f2eb599 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:23.165456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d63b-8b88-4964-8ee5-369d1f2eb599 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:23.165456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418d63b-8b88-4964-8ee5-369d1f2eb599 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:23.166899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:22:23.175768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63e2d7-cb96-4532-bb62-0973fa8bd1c6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:23.175768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63e2d7-cb96-4532-bb62-0973fa8bd1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:23.176793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63e2d7-cb96-4532-bb62-0973fa8bd1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:23.199459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-db63e2d7-cb96-4532-bb62-0973fa8bd1c6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:23.200883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-db63e2d7-cb96-4532-bb62-0973fa8bd1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:23.212003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:23.252165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418d63b-8b88-4964-8ee5-369d1f2eb599 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:23.254183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f418d63b-8b88-4964-8ee5-369d1f2eb599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:23.277784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418d63b-8b88-4964-8ee5-369d1f2eb599 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:23.282016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774e409-e7b6-4bbe-97f6-db8b1dc5b665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:23.282748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774e409-e7b6-4bbe-97f6-db8b1dc5b665 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:23.283039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774e409-e7b6-4bbe-97f6-db8b1dc5b665 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:23.283376 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774e409-e7b6-4bbe-97f6-db8b1dc5b665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:23.293640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774e409-e7b6-4bbe-97f6-db8b1dc5b665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:23.295422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774e409-e7b6-4bbe-97f6-db8b1dc5b665 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:23.306724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 775/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:27.413751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa72e5e-6a98-4563-9b4d-8662961db161 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:27.414342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa72e5e-6a98-4563-9b4d-8662961db161 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:27.414342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa72e5e-6a98-4563-9b4d-8662961db161 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:27.415718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:22:27.421638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba26fc0-3294-4a46-a5f0-90e8dc0ea746 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:27.422735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba26fc0-3294-4a46-a5f0-90e8dc0ea746 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:27.422846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba26fc0-3294-4a46-a5f0-90e8dc0ea746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:27.441624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4ba26fc0-3294-4a46-a5f0-90e8dc0ea746 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:27.442912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4ba26fc0-3294-4a46-a5f0-90e8dc0ea746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:27.451636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:27.745036 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa72e5e-6a98-4563-9b4d-8662961db161 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:27.749752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8aa72e5e-6a98-4563-9b4d-8662961db161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:27.820312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa72e5e-6a98-4563-9b4d-8662961db161 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:27.824139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a298551-94b5-4716-a271-7dbe5ecc48a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:27.824803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a298551-94b5-4716-a271-7dbe5ecc48a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:27.825129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a298551-94b5-4716-a271-7dbe5ecc48a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:27.825403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a298551-94b5-4716-a271-7dbe5ecc48a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:27.836711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a298551-94b5-4716-a271-7dbe5ecc48a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:27.838371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a298551-94b5-4716-a271-7dbe5ecc48a3 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:27.849438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 760/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:22:30.288699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf86fe-2e1a-460a-8250-d94cfcc68c91 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:30.289285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf86fe-2e1a-460a-8250-d94cfcc68c91 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:30.289535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf86fe-2e1a-460a-8250-d94cfcc68c91 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:30.292098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 12:22:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:22:30.299312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fe3e3-c945-453e-989e-175d884ad169 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:30.300142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fe3e3-c945-453e-989e-175d884ad169 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:30.300142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fe3e3-c945-453e-989e-175d884ad169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:30.322730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-da8fe3e3-c945-453e-989e-175d884ad169 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:22:30.324055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-da8fe3e3-c945-453e-989e-175d884ad169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:30.333046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 12:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:22:30.619957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf86fe-2e1a-460a-8250-d94cfcc68c91 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:22:30.621748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7dbf86fe-2e1a-460a-8250-d94cfcc68c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:22:30.630658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=65 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:22:30.636742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf86fe-2e1a-460a-8250-d94cfcc68c91 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:22:30.641295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1d785-490e-4c03-9d86-55d675705d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:22:30.645771 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d785-490e-4c03-9d86-55d675705d94 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:22:30.645771 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d785-490e-4c03-9d86-55d675705d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:22:30.645771 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d785-490e-4c03-9d86-55d675705d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:22:30.652997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1d785-490e-4c03-9d86-55d675705d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:22:30.654509 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1d785-490e-4c03-9d86-55d675705d94 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:22:30.678673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 778/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:22:40.314864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=32 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:22:40.696224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:04.701561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44a831-9b7b-401d-b2b7-d7e31dee88f2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:04.702205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44a831-9b7b-401d-b2b7-d7e31dee88f2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:04.702205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44a831-9b7b-401d-b2b7-d7e31dee88f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:04.733411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ca44a831-9b7b-401d-b2b7-d7e31dee88f2 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:04.735466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ca44a831-9b7b-401d-b2b7-d7e31dee88f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:04.753093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 762/1540] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:04.764297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eea79d-9e91-41a6-9582-8c7a0ed3cb4a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:04.765482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eea79d-9e91-41a6-9582-8c7a0ed3cb4a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:04.766341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eea79d-9e91-41a6-9582-8c7a0ed3cb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:04.803883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-50eea79d-9e91-41a6-9582-8c7a0ed3cb4a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:04.805996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-50eea79d-9e91-41a6-9582-8c7a0ed3cb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:04.825468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 779/1541] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:04.833001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44a831-9b7b-401d-b2b7-d7e31dee88f2 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:04.849391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44a831-9b7b-401d-b2b7-d7e31dee88f2 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:04.879056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6aac59-0915-4c5d-802b-2f974c57c4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:04.880022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6aac59-0915-4c5d-802b-2f974c57c4eb admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:04.880223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6aac59-0915-4c5d-802b-2f974c57c4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:04.880410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6aac59-0915-4c5d-802b-2f974c57c4eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:04.881156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6aac59-0915-4c5d-802b-2f974c57c4eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:04.881496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6aac59-0915-4c5d-802b-2f974c57c4eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:04.883221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6aac59-0915-4c5d-802b-2f974c57c4eb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:04.905280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 763/1542] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:04.921790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eea79d-9e91-41a6-9582-8c7a0ed3cb4a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:04.932779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eea79d-9e91-41a6-9582-8c7a0ed3cb4a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:04.945911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c9916-95d8-4745-843a-e05b174b3752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:04.947033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c9916-95d8-4745-843a-e05b174b3752 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:04.947566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c9916-95d8-4745-843a-e05b174b3752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:04.948062 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c9916-95d8-4745-843a-e05b174b3752 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:04.949015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c9916-95d8-4745-843a-e05b174b3752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:04.951685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c9916-95d8-4745-843a-e05b174b3752 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.052759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 780/1543] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:04 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:05.058889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6aac59-0915-4c5d-802b-2f974c57c4eb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.075139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6aac59-0915-4c5d-802b-2f974c57c4eb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:05.172975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccfcfa-18d8-41d3-9853-016f3a64b890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:05.175930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccfcfa-18d8-41d3-9853-016f3a64b890 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.175930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccfcfa-18d8-41d3-9853-016f3a64b890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.175930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccfcfa-18d8-41d3-9853-016f3a64b890 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.175930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccfcfa-18d8-41d3-9853-016f3a64b890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:05.177275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccfcfa-18d8-41d3-9853-016f3a64b890 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.282162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 764/1544] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:05.286813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c9916-95d8-4745-843a-e05b174b3752 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.301540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c9916-95d8-4745-843a-e05b174b3752 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:05.316936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83cc64-1e05-4af1-a1bf-0682eb5368fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:05.318134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83cc64-1e05-4af1-a1bf-0682eb5368fc admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.318837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83cc64-1e05-4af1-a1bf-0682eb5368fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.319382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83cc64-1e05-4af1-a1bf-0682eb5368fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.320413 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83cc64-1e05-4af1-a1bf-0682eb5368fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:05.327374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83cc64-1e05-4af1-a1bf-0682eb5368fc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.343514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 781/1545] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:05.352327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccfcfa-18d8-41d3-9853-016f3a64b890 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.369414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccfcfa-18d8-41d3-9853-016f3a64b890 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:05.383711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23c7f8-5013-4cb4-9d9e-519a66ccaca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:05.389108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23c7f8-5013-4cb4-9d9e-519a66ccaca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.389798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23c7f8-5013-4cb4-9d9e-519a66ccaca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.390390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23c7f8-5013-4cb4-9d9e-519a66ccaca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.393782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23c7f8-5013-4cb4-9d9e-519a66ccaca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:05.396134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df23c7f8-5013-4cb4-9d9e-519a66ccaca8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.405445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac4fa7-89c5-4dd5-850d-64e17bf03bbf None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.405827 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac4fa7-89c5-4dd5-850d-64e17bf03bbf None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.406121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac4fa7-89c5-4dd5-850d-64e17bf03bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.420564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 765/1546] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:05.422878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561cb08-b6e6-45c0-a716-1f929a1fcf10 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.423118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561cb08-b6e6-45c0-a716-1f929a1fcf10 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.423391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561cb08-b6e6-45c0-a716-1f929a1fcf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.456956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b7ac4fa7-89c5-4dd5-850d-64e17bf03bbf None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.458726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b7ac4fa7-89c5-4dd5-850d-64e17bf03bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.460688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b561cb08-b6e6-45c0-a716-1f929a1fcf10 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.466432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b561cb08-b6e6-45c0-a716-1f929a1fcf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.476260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 782/1547] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.481132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 766/1548] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.481381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a515599-d961-43cc-8dee-25961a91497e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.481690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a515599-d961-43cc-8dee-25961a91497e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.481971 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a515599-d961-43cc-8dee-25961a91497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.486945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561cb08-b6e6-45c0-a716-1f929a1fcf10 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.501880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561cb08-b6e6-45c0-a716-1f929a1fcf10 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:05.507588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee3f9b-9ac9-42eb-bef7-c4bf3832f045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:05.508640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee3f9b-9ac9-42eb-bef7-c4bf3832f045 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.508934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee3f9b-9ac9-42eb-bef7-c4bf3832f045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.509217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee3f9b-9ac9-42eb-bef7-c4bf3832f045 admin admin] PATH_INFO: `/v3/projects/4f4a6b5e282e45ffbef6963200558594/users/707a899d6c5d4c4ea1c58e0de6a85b1d/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.511687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7a515599-d961-43cc-8dee-25961a91497e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.512946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7a515599-d961-43cc-8dee-25961a91497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.522092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 783/1549] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.523734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e8f99-2a60-48a8-9ee7-c01d854ccc38 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.523734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e8f99-2a60-48a8-9ee7-c01d854ccc38 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.524099 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e8f99-2a60-48a8-9ee7-c01d854ccc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.530311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee3f9b-9ac9-42eb-bef7-c4bf3832f045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f4a6b5e282e45ffbef6963200558594, user_id=707a899d6c5d4c4ea1c58e0de6a85b1d, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:05.530311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-2dee3f9b-9ac9-42eb-bef7-c4bf3832f045 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:24:05.530311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 19 12:24:05.530311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: May 19 12:24:05.531226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee3f9b-9ac9-42eb-bef7-c4bf3832f045 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.550173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0a3e8f99-2a60-48a8-9ee7-c01d854ccc38 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.551444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0a3e8f99-2a60-48a8-9ee7-c01d854ccc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.560014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 784/1550] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.561234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1e5aa-a352-40f5-9fca-8386a50c0066 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.561530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1e5aa-a352-40f5-9fca-8386a50c0066 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.561816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1e5aa-a352-40f5-9fca-8386a50c0066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.587591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-04a1e5aa-a352-40f5-9fca-8386a50c0066 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.588805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-04a1e5aa-a352-40f5-9fca-8386a50c0066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.597406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 785/1551] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.599491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a953a-071a-4166-8aaa-6fb518f0c00d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.599782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a953a-071a-4166-8aaa-6fb518f0c00d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.600063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a953a-071a-4166-8aaa-6fb518f0c00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.609048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 767/1552] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:05 2022] PUT /identity/v3/projects/4f4a6b5e282e45ffbef6963200558594/users/707a899d6c5d4c4ea1c58e0de6a85b1d/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:05.611100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91df10d-d3b6-4ad3-9bb8-8fddad621433 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.611394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91df10d-d3b6-4ad3-9bb8-8fddad621433 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.611745 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91df10d-d3b6-4ad3-9bb8-8fddad621433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.620673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cf6a953a-071a-4166-8aaa-6fb518f0c00d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.622165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cf6a953a-071a-4166-8aaa-6fb518f0c00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.630548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f91df10d-d3b6-4ad3-9bb8-8fddad621433 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.631761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f91df10d-d3b6-4ad3-9bb8-8fddad621433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.686212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 786/1553] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.687774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a118028-1523-48b5-9e7d-45bd10ac96d4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.688801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a118028-1523-48b5-9e7d-45bd10ac96d4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.689223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a118028-1523-48b5-9e7d-45bd10ac96d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.700351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 768/1554] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.701667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811876-ab36-40cc-b4f0-589baf36617a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.702256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811876-ab36-40cc-b4f0-589baf36617a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.702256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811876-ab36-40cc-b4f0-589baf36617a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.711955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3a118028-1523-48b5-9e7d-45bd10ac96d4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.713280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3a118028-1523-48b5-9e7d-45bd10ac96d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.721992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 787/1555] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.725044 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017181a3-d83d-4fb1-ac4c-630dbfa5d7af None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.725044 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017181a3-d83d-4fb1-ac4c-630dbfa5d7af None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.725682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-26811876-ab36-40cc-b4f0-589baf36617a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.725851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017181a3-d83d-4fb1-ac4c-630dbfa5d7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.725945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-26811876-ab36-40cc-b4f0-589baf36617a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.740355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 769/1556] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.740355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787101bb-cb8f-47b6-bcc9-93b54175b64b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.740355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787101bb-cb8f-47b6-bcc9-93b54175b64b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.740355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787101bb-cb8f-47b6-bcc9-93b54175b64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.746099 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-017181a3-d83d-4fb1-ac4c-630dbfa5d7af None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.747348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-017181a3-d83d-4fb1-ac4c-630dbfa5d7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.757008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 788/1557] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.758258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d88b1-3ef6-42c3-8bee-76dc1f60b745 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.758583 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d88b1-3ef6-42c3-8bee-76dc1f60b745 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.758864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d88b1-3ef6-42c3-8bee-76dc1f60b745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.769536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-787101bb-cb8f-47b6-bcc9-93b54175b64b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.771272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-787101bb-cb8f-47b6-bcc9-93b54175b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.780506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-523d88b1-3ef6-42c3-8bee-76dc1f60b745 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.782023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-523d88b1-3ef6-42c3-8bee-76dc1f60b745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.782242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 770/1558] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.783553 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f830ca3-ec58-4fe0-aeb5-5f60291695c3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.783850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f830ca3-ec58-4fe0-aeb5-5f60291695c3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.784136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f830ca3-ec58-4fe0-aeb5-5f60291695c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.795905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 789/1559] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.797060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8cc94-0256-441b-b95c-eac92d1e1a51 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.797919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8cc94-0256-441b-b95c-eac92d1e1a51 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.798200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8cc94-0256-441b-b95c-eac92d1e1a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.808782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7f830ca3-ec58-4fe0-aeb5-5f60291695c3 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.810021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7f830ca3-ec58-4fe0-aeb5-5f60291695c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.819340 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-59d8cc94-0256-441b-b95c-eac92d1e1a51 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.819866 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 771/1560] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.820729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-59d8cc94-0256-441b-b95c-eac92d1e1a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.821961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f830ca3-ec58-4fe0-aeb5-5f60291695c3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.835111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 790/1561] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.835379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f830ca3-ec58-4fe0-aeb5-5f60291695c3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:05.836685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8cc94-0256-441b-b95c-eac92d1e1a51 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.841790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f7c8-f75c-47a5-b443-fd60f71f90bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:05.842517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f7c8-f75c-47a5-b443-fd60f71f90bd admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.842808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f7c8-f75c-47a5-b443-fd60f71f90bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.843092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8f7c8-f75c-47a5-b443-fd60f71f90bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.843842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8f7c8-f75c-47a5-b443-fd60f71f90bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:05.844255 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8f7c8-f75c-47a5-b443-fd60f71f90bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:05.846174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb8f7c8-f75c-47a5-b443-fd60f71f90bd admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.849453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:05.850517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8cc94-0256-441b-b95c-eac92d1e1a51 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:05.855648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:05.856484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.856867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.857156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.857928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:05.858245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:05.859378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 12:24:05.859378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 19 12:24:05.859378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: May 19 12:24:05.863028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.863128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 772/1562] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:05.863128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f7c8-f75c-47a5-b443-fd60f71f90bd admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.874123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8f7c8-f75c-47a5-b443-fd60f71f90bd admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:05.880257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7a30c-6ca6-4813-9646-cb5e2981df4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:05.882050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7a30c-6ca6-4813-9646-cb5e2981df4a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.882050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7a30c-6ca6-4813-9646-cb5e2981df4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.882050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7a30c-6ca6-4813-9646-cb5e2981df4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.882576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7a30c-6ca6-4813-9646-cb5e2981df4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:05.882576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7a30c-6ca6-4813-9646-cb5e2981df4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:05.884172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7a30c-6ca6-4813-9646-cb5e2981df4a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.884472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 791/1563] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:05.885314 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.901277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ff0aa-45b0-45c8-bc2b-fb0518d9c316 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:05.906137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 773/1564] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:05.908288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfad54-5cc6-48a2-810f-fa0917a1bdfd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.908588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfad54-5cc6-48a2-810f-fa0917a1bdfd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.908933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfad54-5cc6-48a2-810f-fa0917a1bdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.909349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f5764-aee4-45bb-b5c2-159587b243d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:05.909918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f5764-aee4-45bb-b5c2-159587b243d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.910238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f5764-aee4-45bb-b5c2-159587b243d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.910525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f5764-aee4-45bb-b5c2-159587b243d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.911235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f5764-aee4-45bb-b5c2-159587b243d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:05.911547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f5764-aee4-45bb-b5c2-159587b243d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:05.913168 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f5764-aee4-45bb-b5c2-159587b243d6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.929170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 792/1565] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:05.929170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f5764-aee4-45bb-b5c2-159587b243d6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.931785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-23bfad54-5cc6-48a2-810f-fa0917a1bdfd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:05.934830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-23bfad54-5cc6-48a2-810f-fa0917a1bdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:05.944941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 774/1566] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:05.945283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f5764-aee4-45bb-b5c2-159587b243d6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:05.945761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfad54-5cc6-48a2-810f-fa0917a1bdfd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.952320 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed810b-26f1-456d-8cd4-a78871828d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:05.952994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed810b-26f1-456d-8cd4-a78871828d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:05.953275 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed810b-26f1-456d-8cd4-a78871828d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:05.953594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed810b-26f1-456d-8cd4-a78871828d37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:05.954699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed810b-26f1-456d-8cd4-a78871828d37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:05.955038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed810b-26f1-456d-8cd4-a78871828d37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:05.956729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed810b-26f1-456d-8cd4-a78871828d37 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:05.973319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 793/1567] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:05.975159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed810b-26f1-456d-8cd4-a78871828d37 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:05.997945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed810b-26f1-456d-8cd4-a78871828d37 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.004726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e2665-61d6-4506-9e7f-eeb9eed80619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.005927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e2665-61d6-4506-9e7f-eeb9eed80619 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.012309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e2665-61d6-4506-9e7f-eeb9eed80619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.012309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e2665-61d6-4506-9e7f-eeb9eed80619 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.012309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e2665-61d6-4506-9e7f-eeb9eed80619 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:06.012309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e2665-61d6-4506-9e7f-eeb9eed80619 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.012309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6e2665-61d6-4506-9e7f-eeb9eed80619 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.029555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 794/1568] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:06.030488 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e2665-61d6-4506-9e7f-eeb9eed80619 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.031444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfad54-5cc6-48a2-810f-fa0917a1bdfd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.036287 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5f0e6-4ddc-4dad-8f02-d54e826a049d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.036957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5f0e6-4ddc-4dad-8f02-d54e826a049d service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.037249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5f0e6-4ddc-4dad-8f02-d54e826a049d service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.037533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5f0e6-4ddc-4dad-8f02-d54e826a049d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.045470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e2665-61d6-4506-9e7f-eeb9eed80619 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.051727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5f0e6-4ddc-4dad-8f02-d54e826a049d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.052348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61524-7f3e-4395-98ed-bea548afeb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.052533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61524-7f3e-4395-98ed-bea548afeb8f admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.052882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61524-7f3e-4395-98ed-bea548afeb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.053164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61524-7f3e-4395-98ed-bea548afeb8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.053327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5f0e6-4ddc-4dad-8f02-d54e826a049d service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.053855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb61524-7f3e-4395-98ed-bea548afeb8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.055901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb61524-7f3e-4395-98ed-bea548afeb8f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.067820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 775/1569] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:06.068837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5f0e6-4ddc-4dad-8f02-d54e826a049d service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.071412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:06.080821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5f0e6-4ddc-4dad-8f02-d54e826a049d service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.086988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3753a7-6453-430d-9a72-f5f18ab4ee33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.087652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3753a7-6453-430d-9a72-f5f18ab4ee33 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.088037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3753a7-6453-430d-9a72-f5f18ab4ee33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.088313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3753a7-6453-430d-9a72-f5f18ab4ee33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.089117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3753a7-6453-430d-9a72-f5f18ab4ee33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.090256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-2d3753a7-6453-430d-9a72-f5f18ab4ee33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:24:06.090256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 19 12:24:06.090256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: May 19 12:24:06.091580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3753a7-6453-430d-9a72-f5f18ab4ee33 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.256649 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 795/1570] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:06 2022] POST /identity/v3/projects => generated 405 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:06.260274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61524-7f3e-4395-98ed-bea548afeb8f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.261082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 776/1571] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:06 2022] POST /identity/v3/projects => generated 393 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:06.263196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3753a7-6453-430d-9a72-f5f18ab4ee33 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.356694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3753a7-6453-430d-9a72-f5f18ab4ee33 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.362186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0694bc5-36c5-4f7a-a1f4-8ed381f40445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.363505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0694bc5-36c5-4f7a-a1f4-8ed381f40445 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.364070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0694bc5-36c5-4f7a-a1f4-8ed381f40445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.364384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61524-7f3e-4395-98ed-bea548afeb8f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.364723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0694bc5-36c5-4f7a-a1f4-8ed381f40445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.366468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0694bc5-36c5-4f7a-a1f4-8ed381f40445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.368667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0694bc5-36c5-4f7a-a1f4-8ed381f40445 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.369866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71252b-e3eb-49f7-860f-d3ee15b7a353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.370596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71252b-e3eb-49f7-860f-d3ee15b7a353 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.370962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71252b-e3eb-49f7-860f-d3ee15b7a353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.371230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71252b-e3eb-49f7-860f-d3ee15b7a353 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.371967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71252b-e3eb-49f7-860f-d3ee15b7a353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.373952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f71252b-e3eb-49f7-860f-d3ee15b7a353 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.479251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 796/1572] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:06 2022] POST /identity/v3/projects => generated 389 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:06.481424 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71252b-e3eb-49f7-860f-d3ee15b7a353 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.485524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 777/1573] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:06 2022] POST /identity/v3/projects => generated 383 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:06.487033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0694bc5-36c5-4f7a-a1f4-8ed381f40445 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.585021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:06.615350 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0694bc5-36c5-4f7a-a1f4-8ed381f40445 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.620301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a1144-fe94-43a4-b7bb-a767be76f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.620998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a1144-fe94-43a4-b7bb-a767be76f50f admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.621288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a1144-fe94-43a4-b7bb-a767be76f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.621561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a1144-fe94-43a4-b7bb-a767be76f50f admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.622285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a1144-fe94-43a4-b7bb-a767be76f50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.624356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a1144-fe94-43a4-b7bb-a767be76f50f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.627110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71252b-e3eb-49f7-860f-d3ee15b7a353 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.634129 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0beaa-8285-4bcd-a264-3fba0b788b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.635190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0beaa-8285-4bcd-a264-3fba0b788b1d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.635529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0beaa-8285-4bcd-a264-3fba0b788b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.635917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0beaa-8285-4bcd-a264-3fba0b788b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.636683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0beaa-8285-4bcd-a264-3fba0b788b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:06.636999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0beaa-8285-4bcd-a264-3fba0b788b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.640723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0beaa-8285-4bcd-a264-3fba0b788b1d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.659575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 797/1574] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:06.660467 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0beaa-8285-4bcd-a264-3fba0b788b1d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.673855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0beaa-8285-4bcd-a264-3fba0b788b1d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.680092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f5c8d-e8fc-4755-9e44-ec4ecb04e80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.680805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f5c8d-e8fc-4755-9e44-ec4ecb04e80b admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.681268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f5c8d-e8fc-4755-9e44-ec4ecb04e80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.681775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f5c8d-e8fc-4755-9e44-ec4ecb04e80b admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.682795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f5c8d-e8fc-4755-9e44-ec4ecb04e80b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.685900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f5c8d-e8fc-4755-9e44-ec4ecb04e80b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:06.765607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 778/1575] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:06 2022] POST /identity/v3/projects => generated 365 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:06.770821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a1144-fe94-43a4-b7bb-a767be76f50f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.832593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 798/1576] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:06 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:06.836474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f5c8d-e8fc-4755-9e44-ec4ecb04e80b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:06.903822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a1144-fe94-43a4-b7bb-a767be76f50f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.938863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:06.969908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f5c8d-e8fc-4755-9e44-ec4ecb04e80b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:06.976845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82896cbc-ea9a-4bc4-a09c-d8155f2dfe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:06.977548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82896cbc-ea9a-4bc4-a09c-d8155f2dfe49 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:06.977856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82896cbc-ea9a-4bc4-a09c-d8155f2dfe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:06.978107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82896cbc-ea9a-4bc4-a09c-d8155f2dfe49 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:06.978862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82896cbc-ea9a-4bc4-a09c-d8155f2dfe49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:06.979902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-82896cbc-ea9a-4bc4-a09c-d8155f2dfe49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:24:06.979902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 19 12:24:06.979902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: May 19 12:24:06.981374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82896cbc-ea9a-4bc4-a09c-d8155f2dfe49 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.008431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186ac81-cbfa-4397-8309-9418de591393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.009239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186ac81-cbfa-4397-8309-9418de591393 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.009643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186ac81-cbfa-4397-8309-9418de591393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.010010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186ac81-cbfa-4397-8309-9418de591393 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.010880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186ac81-cbfa-4397-8309-9418de591393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.013082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186ac81-cbfa-4397-8309-9418de591393 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.178322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 799/1577] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:06 2022] POST /identity/v3/users => generated 297 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:07.179725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82896cbc-ea9a-4bc4-a09c-d8155f2dfe49 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.184827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 779/1578] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:06 2022] POST /identity/v3/users => generated 303 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:07.186856 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186ac81-cbfa-4397-8309-9418de591393 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.195192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82896cbc-ea9a-4bc4-a09c-d8155f2dfe49 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.201365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eef6f9-e5eb-441d-a935-9fe01a93f2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.202152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eef6f9-e5eb-441d-a935-9fe01a93f2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.202581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eef6f9-e5eb-441d-a935-9fe01a93f2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.204163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eef6f9-e5eb-441d-a935-9fe01a93f2b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.205756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eef6f9-e5eb-441d-a935-9fe01a93f2b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.209028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eef6f9-e5eb-441d-a935-9fe01a93f2b6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.216658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186ac81-cbfa-4397-8309-9418de591393 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.224734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dab23e-eb1e-4e4c-87f0-e8e3b3569a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.225853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dab23e-eb1e-4e4c-87f0-e8e3b3569a96 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.226327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dab23e-eb1e-4e4c-87f0-e8e3b3569a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.226836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dab23e-eb1e-4e4c-87f0-e8e3b3569a96 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.228084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dab23e-eb1e-4e4c-87f0-e8e3b3569a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.231711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dab23e-eb1e-4e4c-87f0-e8e3b3569a96 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.325597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 800/1579] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:07 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:07.326627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eef6f9-e5eb-441d-a935-9fe01a93f2b6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.339495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eef6f9-e5eb-441d-a935-9fe01a93f2b6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.345330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bae9c5-8333-40df-9f81-dea5cbe75262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.346022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bae9c5-8333-40df-9f81-dea5cbe75262 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.346307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bae9c5-8333-40df-9f81-dea5cbe75262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.346619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bae9c5-8333-40df-9f81-dea5cbe75262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.347349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bae9c5-8333-40df-9f81-dea5cbe75262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.349357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bae9c5-8333-40df-9f81-dea5cbe75262 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.358154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 780/1580] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:07 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:07.359100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dab23e-eb1e-4e4c-87f0-e8e3b3569a96 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.374473 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dab23e-eb1e-4e4c-87f0-e8e3b3569a96 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.380528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d0d03-08e2-4ab9-8149-a93c3b2b9eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.381438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d0d03-08e2-4ab9-8149-a93c3b2b9eec admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.381827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d0d03-08e2-4ab9-8149-a93c3b2b9eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.382205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d0d03-08e2-4ab9-8149-a93c3b2b9eec admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.383067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d0d03-08e2-4ab9-8149-a93c3b2b9eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.385432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d0d03-08e2-4ab9-8149-a93c3b2b9eec admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.471637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 781/1581] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:07 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:07.472740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 801/1582] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:07 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:07.474538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bae9c5-8333-40df-9f81-dea5cbe75262 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.479233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d0d03-08e2-4ab9-8149-a93c3b2b9eec admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.498432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d0d03-08e2-4ab9-8149-a93c3b2b9eec admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.546598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:07.589941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004579a-7df6-4c3d-8601-c784ad0e6de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.591048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004579a-7df6-4c3d-8601-c784ad0e6de3 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.591678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004579a-7df6-4c3d-8601-c784ad0e6de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.592147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004579a-7df6-4c3d-8601-c784ad0e6de3 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.593026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004579a-7df6-4c3d-8601-c784ad0e6de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.596766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004579a-7df6-4c3d-8601-c784ad0e6de3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.598237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bae9c5-8333-40df-9f81-dea5cbe75262 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.604843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad678c-0754-4dcf-a038-d5f7fe897f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.605872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad678c-0754-4dcf-a038-d5f7fe897f1c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.606466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad678c-0754-4dcf-a038-d5f7fe897f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.606886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad678c-0754-4dcf-a038-d5f7fe897f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.608208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad678c-0754-4dcf-a038-d5f7fe897f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.610830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad678c-0754-4dcf-a038-d5f7fe897f1c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.622001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 802/1583] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.623066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad678c-0754-4dcf-a038-d5f7fe897f1c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.634117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad678c-0754-4dcf-a038-d5f7fe897f1c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.638801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df967d9-db37-48f1-a3b0-3fd04a5b8511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.639458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df967d9-db37-48f1-a3b0-3fd04a5b8511 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.639743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df967d9-db37-48f1-a3b0-3fd04a5b8511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.640022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df967d9-db37-48f1-a3b0-3fd04a5b8511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.640749 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df967d9-db37-48f1-a3b0-3fd04a5b8511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.642960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df967d9-db37-48f1-a3b0-3fd04a5b8511 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.653456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 803/1584] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.654636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df967d9-db37-48f1-a3b0-3fd04a5b8511 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.666000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df967d9-db37-48f1-a3b0-3fd04a5b8511 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.671064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9a809-5c3e-4526-aae2-ed5e3b336f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.671725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a809-5c3e-4526-aae2-ed5e3b336f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.672086 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a809-5c3e-4526-aae2-ed5e3b336f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.672400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a809-5c3e-4526-aae2-ed5e3b336f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.673384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a809-5c3e-4526-aae2-ed5e3b336f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.675014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a809-5c3e-4526-aae2-ed5e3b336f54 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.689762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 804/1585] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.689762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9a809-5c3e-4526-aae2-ed5e3b336f54 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.696023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 782/1586] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:07 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:07.696865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004579a-7df6-4c3d-8601-c784ad0e6de3 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.701149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9a809-5c3e-4526-aae2-ed5e3b336f54 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.707824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e127e5c-a08e-46ae-81f2-1b13d7f49bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.709043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e127e5c-a08e-46ae-81f2-1b13d7f49bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.709043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e127e5c-a08e-46ae-81f2-1b13d7f49bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.709292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004579a-7df6-4c3d-8601-c784ad0e6de3 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.709396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e127e5c-a08e-46ae-81f2-1b13d7f49bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.709813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e127e5c-a08e-46ae-81f2-1b13d7f49bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.711831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e127e5c-a08e-46ae-81f2-1b13d7f49bd5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.713765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f15fc8-7100-4700-9441-0421b3a58d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.714455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f15fc8-7100-4700-9441-0421b3a58d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.714746 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f15fc8-7100-4700-9441-0421b3a58d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.715026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f15fc8-7100-4700-9441-0421b3a58d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.715754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f15fc8-7100-4700-9441-0421b3a58d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.717362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f15fc8-7100-4700-9441-0421b3a58d9b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.723142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 805/1587] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.724030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e127e5c-a08e-46ae-81f2-1b13d7f49bd5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.731231 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 783/1588] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.732207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f15fc8-7100-4700-9441-0421b3a58d9b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.736697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e127e5c-a08e-46ae-81f2-1b13d7f49bd5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.742513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f15fc8-7100-4700-9441-0421b3a58d9b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.744494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180124a-2014-4dee-99b4-04d969124f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.745433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180124a-2014-4dee-99b4-04d969124f7a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.745649 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180124a-2014-4dee-99b4-04d969124f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.745955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180124a-2014-4dee-99b4-04d969124f7a admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.746740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180124a-2014-4dee-99b4-04d969124f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.748405 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a1f4a-6377-49ed-aed5-3c4114c67f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.748705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180124a-2014-4dee-99b4-04d969124f7a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.749264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a1f4a-6377-49ed-aed5-3c4114c67f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.749546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a1f4a-6377-49ed-aed5-3c4114c67f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.749825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a1f4a-6377-49ed-aed5-3c4114c67f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.751256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a1f4a-6377-49ed-aed5-3c4114c67f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.752837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a1f4a-6377-49ed-aed5-3c4114c67f59 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.770564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 784/1589] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.772313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a1f4a-6377-49ed-aed5-3c4114c67f59 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.786716 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a1f4a-6377-49ed-aed5-3c4114c67f59 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.792588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529edbe-ee7d-4cce-aef1-3d6d9eb045c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.793437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529edbe-ee7d-4cce-aef1-3d6d9eb045c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.793752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529edbe-ee7d-4cce-aef1-3d6d9eb045c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.794167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529edbe-ee7d-4cce-aef1-3d6d9eb045c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.795305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529edbe-ee7d-4cce-aef1-3d6d9eb045c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.796958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529edbe-ee7d-4cce-aef1-3d6d9eb045c9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.818049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 785/1590] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.818049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529edbe-ee7d-4cce-aef1-3d6d9eb045c9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.831729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529edbe-ee7d-4cce-aef1-3d6d9eb045c9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.836536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142b2e1-a204-42ec-99b4-57d1df0f98a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.837220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142b2e1-a204-42ec-99b4-57d1df0f98a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.837500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142b2e1-a204-42ec-99b4-57d1df0f98a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.837778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142b2e1-a204-42ec-99b4-57d1df0f98a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.838539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142b2e1-a204-42ec-99b4-57d1df0f98a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.840154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142b2e1-a204-42ec-99b4-57d1df0f98a1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.854658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 786/1591] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.854658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142b2e1-a204-42ec-99b4-57d1df0f98a1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.866884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 806/1592] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:07 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:07.868292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180124a-2014-4dee-99b4-04d969124f7a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.873316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142b2e1-a204-42ec-99b4-57d1df0f98a1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.879891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7786e-1796-4691-93e7-c4629fc05171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.881812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7786e-1796-4691-93e7-c4629fc05171 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.881812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7786e-1796-4691-93e7-c4629fc05171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.881812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7786e-1796-4691-93e7-c4629fc05171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.882967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7786e-1796-4691-93e7-c4629fc05171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.884966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7786e-1796-4691-93e7-c4629fc05171 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.901513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180124a-2014-4dee-99b4-04d969124f7a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.901993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 787/1593] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.903202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7786e-1796-4691-93e7-c4629fc05171 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.907899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f831a6-81cd-4d4e-a6d5-af8be4785c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.908780 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f831a6-81cd-4d4e-a6d5-af8be4785c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.909305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f831a6-81cd-4d4e-a6d5-af8be4785c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.909619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f831a6-81cd-4d4e-a6d5-af8be4785c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.911156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f831a6-81cd-4d4e-a6d5-af8be4785c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.915662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f831a6-81cd-4d4e-a6d5-af8be4785c15 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.916911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7786e-1796-4691-93e7-c4629fc05171 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.926652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d592f-3aa7-40d4-bd6f-f644dcd272a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.927357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d592f-3aa7-40d4-bd6f-f644dcd272a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.927639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d592f-3aa7-40d4-bd6f-f644dcd272a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.927926 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d592f-3aa7-40d4-bd6f-f644dcd272a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.928661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d592f-3aa7-40d4-bd6f-f644dcd272a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.930472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d592f-3aa7-40d4-bd6f-f644dcd272a0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.936197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 807/1594] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.937485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f831a6-81cd-4d4e-a6d5-af8be4785c15 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.947424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 788/1595] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:07.948182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d592f-3aa7-40d4-bd6f-f644dcd272a0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:07.953516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f831a6-81cd-4d4e-a6d5-af8be4785c15 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.958807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ec8b4-ea50-4186-9f32-755c74e4329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.959729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ec8b4-ea50-4186-9f32-755c74e4329a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.960005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ec8b4-ea50-4186-9f32-755c74e4329a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.960476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ec8b4-ea50-4186-9f32-755c74e4329a admin admin] PATH_INFO: `/v3/projects/d9e96ed4a3b5430298605d709d353f6e/users/5c35bd3640694b0dbce13c713fd68306/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.960728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d592f-3aa7-40d4-bd6f-f644dcd272a0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:07.965991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf611a8-1780-4c7e-a11e-93b2df037c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:07.966868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf611a8-1780-4c7e-a11e-93b2df037c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:07.967151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf611a8-1780-4c7e-a11e-93b2df037c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:07.967423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf611a8-1780-4c7e-a11e-93b2df037c60 admin admin] PATH_INFO: `/v3/projects/278a14fbad8b40b897b03390c4d55851/users/4214f385b53c4577a80d9781a74011b8/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:07.978724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ec8b4-ea50-4186-9f32-755c74e4329a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e96ed4a3b5430298605d709d353f6e, user_id=5c35bd3640694b0dbce13c713fd68306, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.981548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ec8b4-ea50-4186-9f32-755c74e4329a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:07.991946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf611a8-1780-4c7e-a11e-93b2df037c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278a14fbad8b40b897b03390c4d55851, user_id=4214f385b53c4577a80d9781a74011b8, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:07.996915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf611a8-1780-4c7e-a11e-93b2df037c60 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:08.086717 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 808/1596] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:07 2022] PUT /identity/v3/projects/d9e96ed4a3b5430298605d709d353f6e/users/5c35bd3640694b0dbce13c713fd68306/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:08.088132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ec8b4-ea50-4186-9f32-755c74e4329a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:08.104669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:08.156248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 789/1597] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:07 2022] PUT /identity/v3/projects/278a14fbad8b40b897b03390c4d55851/users/4214f385b53c4577a80d9781a74011b8/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:08.157593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf611a8-1780-4c7e-a11e-93b2df037c60 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:08.177934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ec8b4-ea50-4186-9f32-755c74e4329a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:08.261052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf611a8-1780-4c7e-a11e-93b2df037c60 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:08.272485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f41040-005d-48c7-9eba-4ea68fda6f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:08.277391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f41040-005d-48c7-9eba-4ea68fda6f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.278824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f41040-005d-48c7-9eba-4ea68fda6f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.284492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f41040-005d-48c7-9eba-4ea68fda6f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.284492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f41040-005d-48c7-9eba-4ea68fda6f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:08.284492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f41040-005d-48c7-9eba-4ea68fda6f1a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:08.288941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97371281-127e-42b0-8fa4-8b3da8150392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:08.289977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97371281-127e-42b0-8fa4-8b3da8150392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.290322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97371281-127e-42b0-8fa4-8b3da8150392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.290760 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97371281-127e-42b0-8fa4-8b3da8150392 admin admin] PATH_INFO: `/v3/projects/86c353f6ded643e6b501ec57800fafc6/users/f0a98f1c068843a684f6765d41cc4e49/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.312552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97371281-127e-42b0-8fa4-8b3da8150392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c353f6ded643e6b501ec57800fafc6, user_id=f0a98f1c068843a684f6765d41cc4e49, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:08.313625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 790/1598] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:08.315381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f41040-005d-48c7-9eba-4ea68fda6f1a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:08.320835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97371281-127e-42b0-8fa4-8b3da8150392 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:08.332137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f41040-005d-48c7-9eba-4ea68fda6f1a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:08.337269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47434f96-5b55-4017-b1b8-0138d7da0ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:08.338026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47434f96-5b55-4017-b1b8-0138d7da0ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.338392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47434f96-5b55-4017-b1b8-0138d7da0ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.338701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47434f96-5b55-4017-b1b8-0138d7da0ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.339451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47434f96-5b55-4017-b1b8-0138d7da0ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:08.341101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47434f96-5b55-4017-b1b8-0138d7da0ed4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:08.352444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 791/1599] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:08.353460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47434f96-5b55-4017-b1b8-0138d7da0ed4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:08.365489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47434f96-5b55-4017-b1b8-0138d7da0ed4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:08.370167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c58055-6834-4516-8c88-6d6157b991d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:08.371428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c58055-6834-4516-8c88-6d6157b991d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.371671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c58055-6834-4516-8c88-6d6157b991d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.372079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c58055-6834-4516-8c88-6d6157b991d6 admin admin] PATH_INFO: `/v3/projects/1231585c94524ec199c47bd4f4162578/users/746a70534fe5419ebe6d5e97a7a041f1/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.385387 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:08.389527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c58055-6834-4516-8c88-6d6157b991d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1231585c94524ec199c47bd4f4162578, user_id=746a70534fe5419ebe6d5e97a7a041f1, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:08.394080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c58055-6834-4516-8c88-6d6157b991d6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:08.449508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 809/1600] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:08 2022] PUT /identity/v3/projects/86c353f6ded643e6b501ec57800fafc6/users/f0a98f1c068843a684f6765d41cc4e49/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:08.450491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97371281-127e-42b0-8fa4-8b3da8150392 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:08.496321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 792/1601] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:08 2022] PUT /identity/v3/projects/1231585c94524ec199c47bd4f4162578/users/746a70534fe5419ebe6d5e97a7a041f1/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:08.500935 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c844b-c1ef-40aa-9827-7c268136f9ea None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.501226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c844b-c1ef-40aa-9827-7c268136f9ea None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.501755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c844b-c1ef-40aa-9827-7c268136f9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.522232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1e2c844b-c1ef-40aa-9827-7c268136f9ea None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:08.523901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1e2c844b-c1ef-40aa-9827-7c268136f9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:08.524053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97371281-127e-42b0-8fa4-8b3da8150392 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:08.596202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:08.606925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 793/1602] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:08.609027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e411-c6c2-4fea-965d-29a1a2092148 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.609422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e411-c6c2-4fea-965d-29a1a2092148 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.609805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45e411-c6c2-4fea-965d-29a1a2092148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.617921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d032b8-6110-4938-ae0f-affc0b8eb5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:08.618877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d032b8-6110-4938-ae0f-affc0b8eb5a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.619162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d032b8-6110-4938-ae0f-affc0b8eb5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.619429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d032b8-6110-4938-ae0f-affc0b8eb5a5 admin admin] PATH_INFO: `/v3/projects/a332bbbc8cd04a7ab5b13aa33ef7a783/users/8fe038c184634d46b45c52ab9d23eb4d/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.645901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-be45e411-c6c2-4fea-965d-29a1a2092148 None None] MFA Rules not processed for user `4214f385b53c4577a80d9781a74011b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:08.647530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d032b8-6110-4938-ae0f-affc0b8eb5a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a332bbbc8cd04a7ab5b13aa33ef7a783, user_id=8fe038c184634d46b45c52ab9d23eb4d, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:08.647756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-be45e411-c6c2-4fea-965d-29a1a2092148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:08.653284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d032b8-6110-4938-ae0f-affc0b8eb5a5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:08.729414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 794/1603] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:08.730940 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e411-c6c2-4fea-965d-29a1a2092148 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:08.745766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45e411-c6c2-4fea-965d-29a1a2092148 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:08.753207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0ba05-ad5b-4f85-bef0-2fcae4120b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:08.754052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0ba05-ad5b-4f85-bef0-2fcae4120b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.754340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0ba05-ad5b-4f85-bef0-2fcae4120b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.754725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0ba05-ad5b-4f85-bef0-2fcae4120b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.755548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d0ba05-ad5b-4f85-bef0-2fcae4120b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:08.757240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d0ba05-ad5b-4f85-bef0-2fcae4120b0e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:08.771440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 795/1604] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:08.772894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0ba05-ad5b-4f85-bef0-2fcae4120b0e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:08.773818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 810/1605] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:08 2022] PUT /identity/v3/projects/a332bbbc8cd04a7ab5b13aa33ef7a783/users/8fe038c184634d46b45c52ab9d23eb4d/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:08.776115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81967c5-1ede-4f94-abac-a09014aa23e2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.776676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81967c5-1ede-4f94-abac-a09014aa23e2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.777141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81967c5-1ede-4f94-abac-a09014aa23e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.787123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0ba05-ad5b-4f85-bef0-2fcae4120b0e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:08.804273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b81967c5-1ede-4f94-abac-a09014aa23e2 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:08.805606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b81967c5-1ede-4f94-abac-a09014aa23e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:08.857043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3bec3-bca4-430a-a676-7a48c36c5342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:08.860947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3bec3-bca4-430a-a676-7a48c36c5342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.861268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3bec3-bca4-430a-a676-7a48c36c5342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.861552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3bec3-bca4-430a-a676-7a48c36c5342 admin admin] PATH_INFO: `/v3/projects/8484f3ad67724bc2bd59dd170e23d7ee/users/4ad0bd03fd2e4d19bbce4c31096d69c8/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.879072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:08.885108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3bec3-bca4-430a-a676-7a48c36c5342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8484f3ad67724bc2bd59dd170e23d7ee, user_id=4ad0bd03fd2e4d19bbce4c31096d69c8, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:08.889967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3bec3-bca4-430a-a676-7a48c36c5342 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:08.896335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 811/1606] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:08.897710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a528b71-1cb0-4ea3-a89c-972aaf0777c0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:08.898176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a528b71-1cb0-4ea3-a89c-972aaf0777c0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:08.898547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a528b71-1cb0-4ea3-a89c-972aaf0777c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:08.929388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4a528b71-1cb0-4ea3-a89c-972aaf0777c0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:08.931312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4a528b71-1cb0-4ea3-a89c-972aaf0777c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:08.942380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 812/1607] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:08.943631 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a528b71-1cb0-4ea3-a89c-972aaf0777c0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:09.039734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 796/1608] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:08 2022] PUT /identity/v3/projects/8484f3ad67724bc2bd59dd170e23d7ee/users/4ad0bd03fd2e4d19bbce4c31096d69c8/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:09.039734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3bec3-bca4-430a-a676-7a48c36c5342 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:09.051864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a528b71-1cb0-4ea3-a89c-972aaf0777c0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:09.133517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada08295-007e-41cf-a19a-7984690b345b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:09.134843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08295-007e-41cf-a19a-7984690b345b service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.135369 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08295-007e-41cf-a19a-7984690b345b service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.135826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08295-007e-41cf-a19a-7984690b345b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.141686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3bec3-bca4-430a-a676-7a48c36c5342 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:09.147959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344304e4-c770-45c2-a317-cee32a7aad28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:09.148915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344304e4-c770-45c2-a317-cee32a7aad28 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.149376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344304e4-c770-45c2-a317-cee32a7aad28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.149823 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344304e4-c770-45c2-a317-cee32a7aad28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.150729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344304e4-c770-45c2-a317-cee32a7aad28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:09.153201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344304e4-c770-45c2-a317-cee32a7aad28 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:09.168834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada08295-007e-41cf-a19a-7984690b345b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:09.173353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada08295-007e-41cf-a19a-7984690b345b service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:09.219643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 813/1609] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:09.220373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada08295-007e-41cf-a19a-7984690b345b service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:09.248917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada08295-007e-41cf-a19a-7984690b345b service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:09.258232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f976b8-b731-49d5-8689-d2bb66445586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:09.258798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f976b8-b731-49d5-8689-d2bb66445586 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.259091 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f976b8-b731-49d5-8689-d2bb66445586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.260417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f976b8-b731-49d5-8689-d2bb66445586 admin admin] PATH_INFO: `/v3/projects/99c98b6b528545f7a18638d5a0f69bb4/users/0bcf2df379a44e4bb05c21c495c38483/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.286499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 797/1610] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:09 2022] POST /identity/v3/projects => generated 403 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:09.286499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fadffc-4378-4bfb-a17b-334928519c1d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.296428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fadffc-4378-4bfb-a17b-334928519c1d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.296679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fadffc-4378-4bfb-a17b-334928519c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.308807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f976b8-b731-49d5-8689-d2bb66445586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c98b6b528545f7a18638d5a0f69bb4, user_id=0bcf2df379a44e4bb05c21c495c38483, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:09.315687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f976b8-b731-49d5-8689-d2bb66445586 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:09.365762 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-56fadffc-4378-4bfb-a17b-334928519c1d None None] MFA Rules not processed for user `8fe038c184634d46b45c52ab9d23eb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:09.367323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-56fadffc-4378-4bfb-a17b-334928519c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:09.413877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:09.426514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 814/1611] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:09 2022] PUT /identity/v3/projects/99c98b6b528545f7a18638d5a0f69bb4/users/0bcf2df379a44e4bb05c21c495c38483/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:09.426514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f976b8-b731-49d5-8689-d2bb66445586 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:09.496954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 798/1612] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:09.498242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fadffc-4378-4bfb-a17b-334928519c1d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:09.533263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f976b8-b731-49d5-8689-d2bb66445586 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:09.538666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2881404-26d4-407e-9dbc-5196aa3add01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:09.539808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2881404-26d4-407e-9dbc-5196aa3add01 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.540305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2881404-26d4-407e-9dbc-5196aa3add01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.540834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2881404-26d4-407e-9dbc-5196aa3add01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.572580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fadffc-4378-4bfb-a17b-334928519c1d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:09.577240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5931-9dfd-48e0-a4ef-45c41072d3d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:09.580195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5931-9dfd-48e0-a4ef-45c41072d3d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.580895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5931-9dfd-48e0-a4ef-45c41072d3d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.581533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5931-9dfd-48e0-a4ef-45c41072d3d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.617982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:09.702846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2881404-26d4-407e-9dbc-5196aa3add01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:09.705230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2881404-26d4-407e-9dbc-5196aa3add01 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:09.712327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb5931-9dfd-48e0-a4ef-45c41072d3d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:09.714120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb5931-9dfd-48e0-a4ef-45c41072d3d7 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:09.721735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 815/1613] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:09.727671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f4e09-b6c1-430b-b912-028df983efae None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.727671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f4e09-b6c1-430b-b912-028df983efae None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.727671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f4e09-b6c1-430b-b912-028df983efae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.728789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 799/1614] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:09.730006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5931-9dfd-48e0-a4ef-45c41072d3d7 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:09.756625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5931-9dfd-48e0-a4ef-45c41072d3d7 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:09.762657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5d5f4e09-b6c1-430b-b912-028df983efae None None] MFA Rules not processed for user `4ad0bd03fd2e4d19bbce4c31096d69c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:09.773620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5d5f4e09-b6c1-430b-b912-028df983efae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:09.774027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bbdc4-c19b-412b-8e55-f1a2e4ec34cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:09.774027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bbdc4-c19b-412b-8e55-f1a2e4ec34cb admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.774027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bbdc4-c19b-412b-8e55-f1a2e4ec34cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.774027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bbdc4-c19b-412b-8e55-f1a2e4ec34cb admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.775671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bbdc4-c19b-412b-8e55-f1a2e4ec34cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:09.780776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bbdc4-c19b-412b-8e55-f1a2e4ec34cb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:09.896656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 816/1615] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:09.899804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8841a-8328-4254-841b-de9d82652bea None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.900579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8841a-8328-4254-841b-de9d82652bea None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.900579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8841a-8328-4254-841b-de9d82652bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.924681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-49d8841a-8328-4254-841b-de9d82652bea None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:09.926641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-49d8841a-8328-4254-841b-de9d82652bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:09.941133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 817/1616] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:09.941133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8841a-8328-4254-841b-de9d82652bea None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:09.947568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 800/1617] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:09 2022] POST /identity/v3/users => generated 308 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:09.948893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bbdc4-c19b-412b-8e55-f1a2e4ec34cb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:09.973442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bbdc4-c19b-412b-8e55-f1a2e4ec34cb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:09.988580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bd960-11c1-4695-b7ba-af8b4f5bff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:09.989933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bd960-11c1-4695-b7ba-af8b4f5bff48 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:09.990464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bd960-11c1-4695-b7ba-af8b4f5bff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:09.994978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bd960-11c1-4695-b7ba-af8b4f5bff48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:09.994978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bd960-11c1-4695-b7ba-af8b4f5bff48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:09.994978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148bd960-11c1-4695-b7ba-af8b4f5bff48 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:10.068362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8841a-8328-4254-841b-de9d82652bea None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:10.080324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9563dc-8a6b-471f-9c20-e7ef68fed243 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:10.085461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9563dc-8a6b-471f-9c20-e7ef68fed243 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:10.085461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9563dc-8a6b-471f-9c20-e7ef68fed243 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:10.085461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9563dc-8a6b-471f-9c20-e7ef68fed243 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:10.174533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 801/1618] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:09 2022] POST /identity/v3/projects => generated 369 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:10.175792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bd960-11c1-4695-b7ba-af8b4f5bff48 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:10.213345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:10.249045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9563dc-8a6b-471f-9c20-e7ef68fed243 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:10.252019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9563dc-8a6b-471f-9c20-e7ef68fed243 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:10.267522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 818/1619] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:10.269428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9563dc-8a6b-471f-9c20-e7ef68fed243 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:10.282724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9563dc-8a6b-471f-9c20-e7ef68fed243 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:10.410119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bd960-11c1-4695-b7ba-af8b4f5bff48 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:10.410558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635752b-d198-4d94-8b90-4774399eec90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:10.411582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635752b-d198-4d94-8b90-4774399eec90 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:10.412023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635752b-d198-4d94-8b90-4774399eec90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:10.413047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635752b-d198-4d94-8b90-4774399eec90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:10.425358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fcd0f-1281-4966-a042-61cc58c609da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:10.426319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fcd0f-1281-4966-a042-61cc58c609da admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:10.426786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fcd0f-1281-4966-a042-61cc58c609da admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:10.427237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fcd0f-1281-4966-a042-61cc58c609da admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:10.428806 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fcd0f-1281-4966-a042-61cc58c609da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:10.434063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fcd0f-1281-4966-a042-61cc58c609da admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:10.451567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635752b-d198-4d94-8b90-4774399eec90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:10.457213 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 802/1620] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:10.458508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fcd0f-1281-4966-a042-61cc58c609da admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:10.462085 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635752b-d198-4d94-8b90-4774399eec90 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:10.482601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fcd0f-1281-4966-a042-61cc58c609da admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:10.492580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6f37a-8e52-46d5-9230-7d5427e87aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:10.493574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6f37a-8e52-46d5-9230-7d5427e87aef admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:10.493869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6f37a-8e52-46d5-9230-7d5427e87aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:10.494199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6f37a-8e52-46d5-9230-7d5427e87aef admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:10.494971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6f37a-8e52-46d5-9230-7d5427e87aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:10.497465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6f37a-8e52-46d5-9230-7d5427e87aef admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:10.498494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 819/1621] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:10.500968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635752b-d198-4d94-8b90-4774399eec90 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:10.521888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635752b-d198-4d94-8b90-4774399eec90 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:10.532397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba366006-9f53-47ea-9dba-f735cc1862fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:10.533972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba366006-9f53-47ea-9dba-f735cc1862fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:10.534525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba366006-9f53-47ea-9dba-f735cc1862fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:10.538389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba366006-9f53-47ea-9dba-f735cc1862fe admin admin] PATH_INFO: `/v3/projects/c08e087fd5894248910953de8d5f6ae4/users/51683a7646e6421eb0fcf1be0d2b6575/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:10.567682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba366006-9f53-47ea-9dba-f735cc1862fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08e087fd5894248910953de8d5f6ae4, user_id=51683a7646e6421eb0fcf1be0d2b6575, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:10.577562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba366006-9f53-47ea-9dba-f735cc1862fe admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:10.622372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 803/1622] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:10 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:10.628441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6f37a-8e52-46d5-9230-7d5427e87aef admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:10.675544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6f37a-8e52-46d5-9230-7d5427e87aef admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:10.690851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c2739-e646-465a-88df-d06b844f6461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:10.691543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c2739-e646-465a-88df-d06b844f6461 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:10.691828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c2739-e646-465a-88df-d06b844f6461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:10.692084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c2739-e646-465a-88df-d06b844f6461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:10.696952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c2739-e646-465a-88df-d06b844f6461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:10.701675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c2739-e646-465a-88df-d06b844f6461 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:10.721867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 820/1623] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:10 2022] PUT /identity/v3/projects/c08e087fd5894248910953de8d5f6ae4/users/51683a7646e6421eb0fcf1be0d2b6575/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:10.726784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba366006-9f53-47ea-9dba-f735cc1862fe admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:10.757067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 804/1624] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:10.759992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c2739-e646-465a-88df-d06b844f6461 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:10.789371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c2739-e646-465a-88df-d06b844f6461 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:10.861257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:10.880997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4ebce-113d-45d5-b225-927480ddb739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:10.882447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4ebce-113d-45d5-b225-927480ddb739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:10.882990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4ebce-113d-45d5-b225-927480ddb739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:10.883549 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4ebce-113d-45d5-b225-927480ddb739 admin admin] PATH_INFO: `/v3/projects/345a24ac225e42eab20c5f5601d7509c/users/411a1b7163464f0fa4efe13e4dfe5c11/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:10.929091 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba366006-9f53-47ea-9dba-f735cc1862fe admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:10.935950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847090c4-aad3-4bce-aa87-436775e1c6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:10.936857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847090c4-aad3-4bce-aa87-436775e1c6cf admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:10.937435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847090c4-aad3-4bce-aa87-436775e1c6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:10.938772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847090c4-aad3-4bce-aa87-436775e1c6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:10.939690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847090c4-aad3-4bce-aa87-436775e1c6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:10.941926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847090c4-aad3-4bce-aa87-436775e1c6cf admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:10.957130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 821/1625] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:24:10.962999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847090c4-aad3-4bce-aa87-436775e1c6cf admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:10.969156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:10.971825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4ebce-113d-45d5-b225-927480ddb739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345a24ac225e42eab20c5f5601d7509c, user_id=411a1b7163464f0fa4efe13e4dfe5c11, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:10.974932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4ebce-113d-45d5-b225-927480ddb739 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:10.976078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847090c4-aad3-4bce-aa87-436775e1c6cf admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:10.982145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2812e-348d-4de0-be2f-b8e3bc007d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:10.983695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2812e-348d-4de0-be2f-b8e3bc007d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:10.985861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2812e-348d-4de0-be2f-b8e3bc007d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:10.986623 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2812e-348d-4de0-be2f-b8e3bc007d19 admin admin] PATH_INFO: `/v3/projects/c08e087fd5894248910953de8d5f6ae4/users/51683a7646e6421eb0fcf1be0d2b6575/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:10.989105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2812e-348d-4de0-be2f-b8e3bc007d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08e087fd5894248910953de8d5f6ae4, user_id=51683a7646e6421eb0fcf1be0d2b6575, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:10.993193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2812e-348d-4de0-be2f-b8e3bc007d19 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:11.012056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 822/1626] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:10 2022] PUT /identity/v3/projects/c08e087fd5894248910953de8d5f6ae4/users/51683a7646e6421eb0fcf1be0d2b6575/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:11.023869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad424e4c-d0d7-48ad-ac70-975818988a7c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:11.025154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad424e4c-d0d7-48ad-ac70-975818988a7c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:11.025443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad424e4c-d0d7-48ad-ac70-975818988a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:11.114776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ad424e4c-d0d7-48ad-ac70-975818988a7c None None] MFA Rules not processed for user `51683a7646e6421eb0fcf1be0d2b6575`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:11.116038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ad424e4c-d0d7-48ad-ac70-975818988a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:11.159076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 805/1627] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:10 2022] PUT /identity/v3/projects/345a24ac225e42eab20c5f5601d7509c/users/411a1b7163464f0fa4efe13e4dfe5c11/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:11.166703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4ebce-113d-45d5-b225-927480ddb739 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:11.591073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 823/1628] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:11.591073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad424e4c-d0d7-48ad-ac70-975818988a7c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:11.642845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4ebce-113d-45d5-b225-927480ddb739 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:11.642845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d40a46-d9b1-4913-9db6-4e67193c708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:11.642845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d40a46-d9b1-4913-9db6-4e67193c708d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:11.642845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d40a46-d9b1-4913-9db6-4e67193c708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:11.643597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d40a46-d9b1-4913-9db6-4e67193c708d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:11.643597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d40a46-d9b1-4913-9db6-4e67193c708d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:11.643597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d40a46-d9b1-4913-9db6-4e67193c708d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:11.643597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 806/1629] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:11.643597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d40a46-d9b1-4913-9db6-4e67193c708d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:11.643597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d40a46-d9b1-4913-9db6-4e67193c708d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:11.644210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27f9c8-0224-4004-94a2-a6983e903fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:11.644210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27f9c8-0224-4004-94a2-a6983e903fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:11.644210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27f9c8-0224-4004-94a2-a6983e903fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:11.646684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad424e4c-d0d7-48ad-ac70-975818988a7c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:11.647085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27f9c8-0224-4004-94a2-a6983e903fb6 admin admin] PATH_INFO: `/v3/projects/345a24ac225e42eab20c5f5601d7509c/users/411a1b7163464f0fa4efe13e4dfe5c11/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:11.647085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27f9c8-0224-4004-94a2-a6983e903fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345a24ac225e42eab20c5f5601d7509c, user_id=411a1b7163464f0fa4efe13e4dfe5c11, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:11.647085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27f9c8-0224-4004-94a2-a6983e903fb6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:11.647085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 807/1630] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:11 2022] PUT /identity/v3/projects/345a24ac225e42eab20c5f5601d7509c/users/411a1b7163464f0fa4efe13e4dfe5c11/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:11.647085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75721e9b-83b8-4a1e-9c6a-36c8c59379c8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:11.647548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75721e9b-83b8-4a1e-9c6a-36c8c59379c8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:11.647548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75721e9b-83b8-4a1e-9c6a-36c8c59379c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:11.647672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d11af-12a4-4c6e-8a53-14fae8de42bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:11.647672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d11af-12a4-4c6e-8a53-14fae8de42bc service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:11.647672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d11af-12a4-4c6e-8a53-14fae8de42bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:11.647672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d11af-12a4-4c6e-8a53-14fae8de42bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:11.647978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-75721e9b-83b8-4a1e-9c6a-36c8c59379c8 None None] MFA Rules not processed for user `411a1b7163464f0fa4efe13e4dfe5c11`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:11.647978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-75721e9b-83b8-4a1e-9c6a-36c8c59379c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:11.670334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d11af-12a4-4c6e-8a53-14fae8de42bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:11.671829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d11af-12a4-4c6e-8a53-14fae8de42bc service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:11.690722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 824/1631] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:11 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:24:11.723493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 808/1632] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:11.841614 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d11af-12a4-4c6e-8a53-14fae8de42bc service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:11.853518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d11af-12a4-4c6e-8a53-14fae8de42bc service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:11.991833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ec309-911e-40a0-8d97-c03705d74755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:11.992565 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ec309-911e-40a0-8d97-c03705d74755 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:11.993528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ec309-911e-40a0-8d97-c03705d74755 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:11.993825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ec309-911e-40a0-8d97-c03705d74755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:12.011824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ec309-911e-40a0-8d97-c03705d74755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:12.013033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ec309-911e-40a0-8d97-c03705d74755 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:12.025685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 825/1633] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:11 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:12.346849 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:12.471822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75721e9b-83b8-4a1e-9c6a-36c8c59379c8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:12.489706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ec309-911e-40a0-8d97-c03705d74755 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:12.490854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75721e9b-83b8-4a1e-9c6a-36c8c59379c8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:12.504576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0f28-f8b3-43af-bd8a-6d15d56fd808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:12.505694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0f28-f8b3-43af-bd8a-6d15d56fd808 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:12.506199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0f28-f8b3-43af-bd8a-6d15d56fd808 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:12.506734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0f28-f8b3-43af-bd8a-6d15d56fd808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:12.604970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:12.654026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ec309-911e-40a0-8d97-c03705d74755 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:12.669005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:12.673026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:12.675422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:12.676489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] PATH_INFO: `/v3/users/4214f385b53c4577a80d9781a74011b8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:12.678002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0f28-f8b3-43af-bd8a-6d15d56fd808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:12.680093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4214f385b53c4577a80d9781a74011b8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:12.682214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0f28-f8b3-43af-bd8a-6d15d56fd808 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:12.686167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:12.712486 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 809/1634] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:12.951423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4214f385b53c4577a80d9781a74011b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:12.971997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4214f385b53c4577a80d9781a74011b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:12.973203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4214f385b53c4577a80d9781a74011b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:12.974751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4214f385b53c4577a80d9781a74011b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:13.078025 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4214f385b53c4577a80d9781a74011b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:13.137421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 826/1635] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:12 2022] DELETE /identity/v3/users/4214f385b53c4577a80d9781a74011b8 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:24:13.142217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d15e6-7192-48f3-ba5f-7a9e21ea5597 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:13.142544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d15e6-7192-48f3-ba5f-7a9e21ea5597 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:13.142832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d15e6-7192-48f3-ba5f-7a9e21ea5597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:13.173693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d91d15e6-7192-48f3-ba5f-7a9e21ea5597 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:13.179921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d91d15e6-7192-48f3-ba5f-7a9e21ea5597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:13.424203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 810/1636] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:13.454659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:13.458439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:13.592246 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db3bb6-9bea-4cd8-b997-164dceb39d8e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:13.610528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f6fc5-3d82-480b-9b88-0ee331529b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:13.611375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f6fc5-3d82-480b-9b88-0ee331529b73 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:13.611749 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f6fc5-3d82-480b-9b88-0ee331529b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:13.612557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f6fc5-3d82-480b-9b88-0ee331529b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:13.629460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f6fc5-3d82-480b-9b88-0ee331529b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:13.632795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f6fc5-3d82-480b-9b88-0ee331529b73 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:13.639391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:13.656106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 827/1637] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:14.143359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b60ab3-8195-415b-8366-5d53cc38db67 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:14.143921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b60ab3-8195-415b-8366-5d53cc38db67 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:14.144014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b60ab3-8195-415b-8366-5d53cc38db67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:14.208327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-13b60ab3-8195-415b-8366-5d53cc38db67 None None] MFA Rules not processed for user `0bcf2df379a44e4bb05c21c495c38483`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:14.211698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-13b60ab3-8195-415b-8366-5d53cc38db67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:14.384779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 811/1638] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:14.398633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429a3eb-22f2-4f61-9c52-bb1d4444a16d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:14.399626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429a3eb-22f2-4f61-9c52-bb1d4444a16d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:14.400096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429a3eb-22f2-4f61-9c52-bb1d4444a16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:14.450186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b429a3eb-22f2-4f61-9c52-bb1d4444a16d None None] MFA Rules not processed for user `0bcf2df379a44e4bb05c21c495c38483`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:14.452409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b429a3eb-22f2-4f61-9c52-bb1d4444a16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:14.471798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 828/1639] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:14.507838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938758d-7bc8-44e8-85e7-3253985407ea None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:14.508134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938758d-7bc8-44e8-85e7-3253985407ea None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:14.508451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938758d-7bc8-44e8-85e7-3253985407ea None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:14.511210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 812/1640] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 12:24:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:24:14.527113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811dd40-6971-4331-8673-45caf1c7d4eb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:14.527290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811dd40-6971-4331-8673-45caf1c7d4eb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:14.527799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811dd40-6971-4331-8673-45caf1c7d4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:14.573078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0811dd40-6971-4331-8673-45caf1c7d4eb None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:14.574642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0811dd40-6971-4331-8673-45caf1c7d4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:14.706327 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 829/1641] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 12:24:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:14.712703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938758d-7bc8-44e8-85e7-3253985407ea None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:14.754762 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938758d-7bc8-44e8-85e7-3253985407ea None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:14.774545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece434f-ced2-4c9c-98c8-5f8ca5223576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:14.775444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece434f-ced2-4c9c-98c8-5f8ca5223576 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:14.775943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece434f-ced2-4c9c-98c8-5f8ca5223576 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:14.776408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece434f-ced2-4c9c-98c8-5f8ca5223576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:14.799253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece434f-ced2-4c9c-98c8-5f8ca5223576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:14.802080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece434f-ced2-4c9c-98c8-5f8ca5223576 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:14.826305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 813/1642] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:15.302358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b126255-7b3c-4045-94f2-283b190c7850 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:15.307760 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b126255-7b3c-4045-94f2-283b190c7850 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:15.310667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b126255-7b3c-4045-94f2-283b190c7850 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:15.312454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 830/1643] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 19 12:24:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:24:15.323782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8a0ae-5ed1-42a9-9ccf-c26a8f807a13 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:15.325820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8a0ae-5ed1-42a9-9ccf-c26a8f807a13 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:15.325820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8a0ae-5ed1-42a9-9ccf-c26a8f807a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:15.367819 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e1c8a0ae-5ed1-42a9-9ccf-c26a8f807a13 None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:15.369000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e1c8a0ae-5ed1-42a9-9ccf-c26a8f807a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:15.400350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b126255-7b3c-4045-94f2-283b190c7850 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:15.402249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8b126255-7b3c-4045-94f2-283b190c7850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:15.415348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 814/1644] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 19 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:15.421890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5307d-182a-411f-a9a7-21bf798b4f4d None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:15.422232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5307d-182a-411f-a9a7-21bf798b4f4d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:15.428380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5307d-182a-411f-a9a7-21bf798b4f4d None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:15.440538 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 815/1645] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 19 12:24:15 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:24:15.444392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da75eba-aed0-4e39-92a2-6c048cb23cf1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:15.446018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da75eba-aed0-4e39-92a2-6c048cb23cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:15.448407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da75eba-aed0-4e39-92a2-6c048cb23cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:15.456824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b126255-7b3c-4045-94f2-283b190c7850 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:15.481632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:15.482686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:15.482884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:15.483164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] PATH_INFO: `/v3/projects/278a14fbad8b40b897b03390c4d55851` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:15.486948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=278a14fbad8b40b897b03390c4d55851)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:15.488027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: WARNING py.warnings [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:24:15.488027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: warnings.warn(msg) May 19 12:24:15.488027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: May 19 12:24:15.495864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:15.515048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1da75eba-aed0-4e39-92a2-6c048cb23cf1 None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:15.519770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1da75eba-aed0-4e39-92a2-6c048cb23cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:15.561591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 816/1646] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:15.574235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da75eba-aed0-4e39-92a2-6c048cb23cf1 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:15.576640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1da75eba-aed0-4e39-92a2-6c048cb23cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:15.663152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:15.694777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da75eba-aed0-4e39-92a2-6c048cb23cf1 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:15.714077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3b94e-ca63-4de9-a88a-6197dc54c7d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:15.714077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3b94e-ca63-4de9-a88a-6197dc54c7d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:15.714077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3b94e-ca63-4de9-a88a-6197dc54c7d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:15.714077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3b94e-ca63-4de9-a88a-6197dc54c7d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:15.723117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3b94e-ca63-4de9-a88a-6197dc54c7d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:15.739869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3b94e-ca63-4de9-a88a-6197dc54c7d1 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:15.765426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278a14fbad8b40b897b03390c4d55851', 'request_id': 'req-589f2023-5c48-4a53-b720-253b8ace2907'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:15.765426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278a14fbad8b40b897b03390c4d55851', 'request_id': 'req-589f2023-5c48-4a53-b720-253b8ace2907'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:15.768721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 817/1647] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:24:15.770640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c4d89-a5d8-42b1-ad28-18efbe63d9f9 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:15.771300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c4d89-a5d8-42b1-ad28-18efbe63d9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:15.771770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c4d89-a5d8-42b1-ad28-18efbe63d9f9 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:15.773456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 818/1648] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 19 12:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:24:15.788100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b781a-5d24-4425-b71b-c0bbc6e0bfe4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:15.788821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b781a-5d24-4425-b71b-c0bbc6e0bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:15.817431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b781a-5d24-4425-b71b-c0bbc6e0bfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:15.885680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a56b781a-5d24-4425-b71b-c0bbc6e0bfe4 None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:15.891969 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a56b781a-5d24-4425-b71b-c0bbc6e0bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:15.918054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278a14fbad8b40b897b03390c4d55851', 'request_id': 'req-589f2023-5c48-4a53-b720-253b8ace2907'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:15.988297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] The token cache is being invalidate because project 278a14fbad8b40b897b03390c4d55851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:24:15.988834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:15.996991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:16.009420 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 831/1649] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:24:15 2022] DELETE /identity/v3/projects/278a14fbad8b40b897b03390c4d55851 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:16.011179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:16.030207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:16.178018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 819/1650] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:16.202094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b781a-5d24-4425-b71b-c0bbc6e0bfe4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:16.204133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a56b781a-5d24-4425-b71b-c0bbc6e0bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:16.322427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f2023-5c48-4a53-b720-253b8ace2907 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:16.333836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:16.335149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:16.335615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:16.336766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] PATH_INFO: `/v3/users/51683a7646e6421eb0fcf1be0d2b6575` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:16.340049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51683a7646e6421eb0fcf1be0d2b6575)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:16.342830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:16.405291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b781a-5d24-4425-b71b-c0bbc6e0bfe4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:16.410145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66067ca7-a999-42bc-99ec-673a3d9201ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9969632e796345fa9671a45cf61ede3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:16.411765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66067ca7-a999-42bc-99ec-673a3d9201ea service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:16.412060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66067ca7-a999-42bc-99ec-673a3d9201ea service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:16.415496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66067ca7-a999-42bc-99ec-673a3d9201ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:16.429703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66067ca7-a999-42bc-99ec-673a3d9201ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:16.431383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66067ca7-a999-42bc-99ec-673a3d9201ea service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:16.464063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 820/1651] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:16.534960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:16.685175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51683a7646e6421eb0fcf1be0d2b6575'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:16.697510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51683a7646e6421eb0fcf1be0d2b6575'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:16.698863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51683a7646e6421eb0fcf1be0d2b6575'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:16.700280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51683a7646e6421eb0fcf1be0d2b6575'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:16.767441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51683a7646e6421eb0fcf1be0d2b6575'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:16.820498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 832/1652] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:16 2022] DELETE /identity/v3/users/51683a7646e6421eb0fcf1be0d2b6575 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:16.823513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e89993-644d-4a4b-a9c5-84439db8e3c2 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:16.823674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e89993-644d-4a4b-a9c5-84439db8e3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:16.823959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e89993-644d-4a4b-a9c5-84439db8e3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:16.876617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d7e89993-644d-4a4b-a9c5-84439db8e3c2 None None] MFA Rules not processed for user `707a899d6c5d4c4ea1c58e0de6a85b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:16.879209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d7e89993-644d-4a4b-a9c5-84439db8e3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:17.000546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 821/1653] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:17.033560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:17.034950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:17.053540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddce03f-7048-4a5a-99bb-86be2ba09d14 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:17.053843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddce03f-7048-4a5a-99bb-86be2ba09d14 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:17.054130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddce03f-7048-4a5a-99bb-86be2ba09d14 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:17.056047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 822/1654] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 19 12:24:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:24:17.062033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b260d97-b569-41d1-8256-f6fe77d23054 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:17.062325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b260d97-b569-41d1-8256-f6fe77d23054 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:17.062692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b260d97-b569-41d1-8256-f6fe77d23054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:17.074171 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:17.124884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1b260d97-b569-41d1-8256-f6fe77d23054 None None] MFA Rules not processed for user `8b266ea9964c4a179c2743b9cf6478eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:17.127036 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1b260d97-b569-41d1-8256-f6fe77d23054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:17.213343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 823/1655] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:17.234465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b260d97-b569-41d1-8256-f6fe77d23054 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:17.237017 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1b260d97-b569-41d1-8256-f6fe77d23054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:17.256400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd85316-b8e7-49d0-ba25-bc892e643f80 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:17.259027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b260d97-b569-41d1-8256-f6fe77d23054 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:17.272162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff118b7-8ebc-404c-96a2-abe08b9a50a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:17.273760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff118b7-8ebc-404c-96a2-abe08b9a50a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:17.276553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4c2ff-9334-4a7f-a371-4835f19d7ffd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:17.277250 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4c2ff-9334-4a7f-a371-4835f19d7ffd service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:17.277537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4c2ff-9334-4a7f-a371-4835f19d7ffd service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:17.277863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff118b7-8ebc-404c-96a2-abe08b9a50a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:17.286528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff118b7-8ebc-404c-96a2-abe08b9a50a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:17.286894 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4c2ff-9334-4a7f-a371-4835f19d7ffd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:17.298500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff118b7-8ebc-404c-96a2-abe08b9a50a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:17.299369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff118b7-8ebc-404c-96a2-abe08b9a50a6 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:17.306589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4c2ff-9334-4a7f-a371-4835f19d7ffd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:17.310485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4c2ff-9334-4a7f-a371-4835f19d7ffd service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:17.325755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 824/1656] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:17.342180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 833/1657] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:18.759002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961a2f2-0d87-47af-8ae8-a3aacdeac319 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:18.759002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961a2f2-0d87-47af-8ae8-a3aacdeac319 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:18.761068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961a2f2-0d87-47af-8ae8-a3aacdeac319 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:18.763952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 825/1658] 10.222.0.57 () {54 vars in 960 bytes} [Thu May 19 12:24:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:24:18.770460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cad2e-b07e-4b9c-9e79-9adf87a13a41 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:18.771074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cad2e-b07e-4b9c-9e79-9adf87a13a41 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:18.771360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cad2e-b07e-4b9c-9e79-9adf87a13a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:18.788740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961a2f2-0d87-47af-8ae8-a3aacdeac319 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:18.790154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2961a2f2-0d87-47af-8ae8-a3aacdeac319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:18.838590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c38cad2e-b07e-4b9c-9e79-9adf87a13a41 None None] MFA Rules not processed for user `4f41920bad3147bba0000de911dfe5a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:18.841190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c38cad2e-b07e-4b9c-9e79-9adf87a13a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:18.917463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961a2f2-0d87-47af-8ae8-a3aacdeac319 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:18.924282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:18.925510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:18.925939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:18.926357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] PATH_INFO: `/v3/projects/c08e087fd5894248910953de8d5f6ae4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:18.928777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c08e087fd5894248910953de8d5f6ae4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:18.931168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:19.072447 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 834/1659] 10.222.0.57 () {58 vars in 1057 bytes} [Thu May 19 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:19.074473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cad2e-b07e-4b9c-9e79-9adf87a13a41 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:19.106962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cad2e-b07e-4b9c-9e79-9adf87a13a41 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:19.151785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c08e087fd5894248910953de8d5f6ae4', 'request_id': 'req-eee143c2-eab9-4b50-9eac-37cf4aa00374'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:19.168540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c08e087fd5894248910953de8d5f6ae4', 'request_id': 'req-eee143c2-eab9-4b50-9eac-37cf4aa00374'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:19.291883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c08e087fd5894248910953de8d5f6ae4', 'request_id': 'req-eee143c2-eab9-4b50-9eac-37cf4aa00374'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:19.297702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19385b-beaf-4bef-ab4e-a7cc06502fbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:19.299063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19385b-beaf-4bef-ab4e-a7cc06502fbf service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:19.300628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19385b-beaf-4bef-ab4e-a7cc06502fbf service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:19.305359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19385b-beaf-4bef-ab4e-a7cc06502fbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:19.305359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ac19385b-beaf-4bef-ab4e-a7cc06502fbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:19.323723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] The token cache is being invalidate because project c08e087fd5894248910953de8d5f6ae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:24:19.324319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:19.328891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-eee143c2-eab9-4b50-9eac-37cf4aa00374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:19.333615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19385b-beaf-4bef-ab4e-a7cc06502fbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:19.336638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19385b-beaf-4bef-ab4e-a7cc06502fbf service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:19.339467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 826/1660] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:24:18 2022] DELETE /identity/v3/projects/c08e087fd5894248910953de8d5f6ae4 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:19.343394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0390ed-e7ef-449c-aec3-8083ae845893 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:19.343686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0390ed-e7ef-449c-aec3-8083ae845893 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:19.344071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0390ed-e7ef-449c-aec3-8083ae845893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:19.349848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 835/1661] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:19.415563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9b0390ed-e7ef-449c-aec3-8083ae845893 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:19.417063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9b0390ed-e7ef-449c-aec3-8083ae845893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:19.453122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:19.596834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 827/1662] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:19.607870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1fe20-ec56-4a63-a040-db79c3acdab0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:19.608288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1fe20-ec56-4a63-a040-db79c3acdab0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:19.608641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1fe20-ec56-4a63-a040-db79c3acdab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:19.654867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-25f1fe20-ec56-4a63-a040-db79c3acdab0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:19.661462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-25f1fe20-ec56-4a63-a040-db79c3acdab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:19.699513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 836/1663] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:19.701207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0390ed-e7ef-449c-aec3-8083ae845893 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:19.709949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:19.713747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0390ed-e7ef-449c-aec3-8083ae845893 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:19.742971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1536b-b7e2-4d64-b61f-7923c265a202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:19.746813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1536b-b7e2-4d64-b61f-7923c265a202 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:19.749173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1536b-b7e2-4d64-b61f-7923c265a202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:19.749660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1536b-b7e2-4d64-b61f-7923c265a202 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:19.751756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1536b-b7e2-4d64-b61f-7923c265a202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:19.759615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1536b-b7e2-4d64-b61f-7923c265a202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:19.762591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1536b-b7e2-4d64-b61f-7923c265a202 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:19.801212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 828/1664] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:19.801751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1fe20-ec56-4a63-a040-db79c3acdab0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:19.818303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1fe20-ec56-4a63-a040-db79c3acdab0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:19.838564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38944355-96c4-47b9-a033-2f95ae50ede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:19.840451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38944355-96c4-47b9-a033-2f95ae50ede9 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:19.840451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38944355-96c4-47b9-a033-2f95ae50ede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:19.842092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38944355-96c4-47b9-a033-2f95ae50ede9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:19.846737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38944355-96c4-47b9-a033-2f95ae50ede9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:19.849120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38944355-96c4-47b9-a033-2f95ae50ede9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:20.053161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 837/1665] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:19 2022] POST /identity/v3/projects => generated 367 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:24:20.054501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1536b-b7e2-4d64-b61f-7923c265a202 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:20.075525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1536b-b7e2-4d64-b61f-7923c265a202 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:20.228535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff98d1-379e-4710-aa11-4ecd27838593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:20.229233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff98d1-379e-4710-aa11-4ecd27838593 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:20.229519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff98d1-379e-4710-aa11-4ecd27838593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:20.229795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff98d1-379e-4710-aa11-4ecd27838593 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:20.230804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff98d1-379e-4710-aa11-4ecd27838593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:20.234132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff98d1-379e-4710-aa11-4ecd27838593 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:20.408233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38944355-96c4-47b9-a033-2f95ae50ede9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:20.409291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 829/1666] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:20 2022] POST /identity/v3/users => generated 284 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:20.425672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38944355-96c4-47b9-a033-2f95ae50ede9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:20.440117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2c8a5-541b-4154-a8cd-ba5c33166516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:20.441194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2c8a5-541b-4154-a8cd-ba5c33166516 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:20.441739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2c8a5-541b-4154-a8cd-ba5c33166516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:20.442217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2c8a5-541b-4154-a8cd-ba5c33166516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:20.444592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2c8a5-541b-4154-a8cd-ba5c33166516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:20.448485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2c8a5-541b-4154-a8cd-ba5c33166516 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:20.478541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff98d1-379e-4710-aa11-4ecd27838593 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:20.491785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff98d1-379e-4710-aa11-4ecd27838593 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:20.496173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 838/1667] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:20.505556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f149805-cf2c-46c6-8108-31da664ffd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:20.507005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f149805-cf2c-46c6-8108-31da664ffd86 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:20.508677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f149805-cf2c-46c6-8108-31da664ffd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:20.509198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f149805-cf2c-46c6-8108-31da664ffd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:20.511296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f149805-cf2c-46c6-8108-31da664ffd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:20.514475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f149805-cf2c-46c6-8108-31da664ffd86 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:20.557969 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 830/1668] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:24:20.559753 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2c8a5-541b-4154-a8cd-ba5c33166516 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:20.577910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2c8a5-541b-4154-a8cd-ba5c33166516 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:20.623232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3cabc-12bc-41b3-ad56-f8fefb956654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:20.623232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3cabc-12bc-41b3-ad56-f8fefb956654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:20.623232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3cabc-12bc-41b3-ad56-f8fefb956654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:20.623814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3cabc-12bc-41b3-ad56-f8fefb956654 admin admin] PATH_INFO: `/v3/projects/5258d7da4efe4da78362ec8d6b059a39/users/253078990e29460db068751dcab6c8d0/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:20.675297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3cabc-12bc-41b3-ad56-f8fefb956654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5258d7da4efe4da78362ec8d6b059a39, user_id=253078990e29460db068751dcab6c8d0, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:20.679000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3cabc-12bc-41b3-ad56-f8fefb956654 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:20.809893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 839/1669] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:20 2022] PUT /identity/v3/projects/5258d7da4efe4da78362ec8d6b059a39/users/253078990e29460db068751dcab6c8d0/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:20.821888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f20e16-6ab0-488d-9b26-8d7ebf963cfd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:20.822482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f20e16-6ab0-488d-9b26-8d7ebf963cfd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:20.822570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f20e16-6ab0-488d-9b26-8d7ebf963cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:20.889267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-72f20e16-6ab0-488d-9b26-8d7ebf963cfd None None] MFA Rules not processed for user `253078990e29460db068751dcab6c8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:20.890269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-72f20e16-6ab0-488d-9b26-8d7ebf963cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:21.067270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 831/1670] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:21.072884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3cabc-12bc-41b3-ad56-f8fefb956654 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:21.331727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3cabc-12bc-41b3-ad56-f8fefb956654 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:21.359682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cf169-256c-4033-af5c-116806713292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:21.361604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cf169-256c-4033-af5c-116806713292 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:21.363942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cf169-256c-4033-af5c-116806713292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:21.364440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cf169-256c-4033-af5c-116806713292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:21.369550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cf169-256c-4033-af5c-116806713292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:21.371863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0cf169-256c-4033-af5c-116806713292 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:21.565388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 840/1671] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:21 2022] POST /identity/v3/projects => generated 367 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:21.571125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f20e16-6ab0-488d-9b26-8d7ebf963cfd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:21.616639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f20e16-6ab0-488d-9b26-8d7ebf963cfd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:21.750593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa57cd-b371-486a-807a-71c009ce62e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:21.753312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa57cd-b371-486a-807a-71c009ce62e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:21.753674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa57cd-b371-486a-807a-71c009ce62e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:21.754660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa57cd-b371-486a-807a-71c009ce62e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:21.756770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa57cd-b371-486a-807a-71c009ce62e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:21.760530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa57cd-b371-486a-807a-71c009ce62e1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:21.918294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 832/1672] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:21 2022] POST /identity/v3/users => generated 290 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:21.924366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cf169-256c-4033-af5c-116806713292 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:21.942483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cf169-256c-4033-af5c-116806713292 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:21.951501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a81f4-9382-477e-888a-15a426b98276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:21.952280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a81f4-9382-477e-888a-15a426b98276 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:21.952575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a81f4-9382-477e-888a-15a426b98276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:21.952896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a81f4-9382-477e-888a-15a426b98276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:21.954974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a81f4-9382-477e-888a-15a426b98276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:21.956873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a81f4-9382-477e-888a-15a426b98276 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:21.970187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 841/1673] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:21.974349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa57cd-b371-486a-807a-71c009ce62e1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:21.987028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa57cd-b371-486a-807a-71c009ce62e1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:21.994325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc793-df2f-4a2f-8d52-2f2736d8789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:21.995909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc793-df2f-4a2f-8d52-2f2736d8789e admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:21.996423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc793-df2f-4a2f-8d52-2f2736d8789e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:21.997370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc793-df2f-4a2f-8d52-2f2736d8789e admin admin] PATH_INFO: `/v3/projects/8437cc6af4ed477f88dbea8d1ec38297/users/8bc15f227e2d48289bc0f30210291993/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:22.019250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdc793-df2f-4a2f-8d52-2f2736d8789e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8437cc6af4ed477f88dbea8d1ec38297, user_id=8bc15f227e2d48289bc0f30210291993, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:22.023258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdc793-df2f-4a2f-8d52-2f2736d8789e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:22.210259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:22.221931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 833/1674] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:21 2022] PUT /identity/v3/projects/8437cc6af4ed477f88dbea8d1ec38297/users/8bc15f227e2d48289bc0f30210291993/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:22.225887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a81f4-9382-477e-888a-15a426b98276 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:22.245531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a81f4-9382-477e-888a-15a426b98276 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:22.435272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc3867-c2b1-4c8f-a484-35bc1f6a7093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:22.435983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3867-c2b1-4c8f-a484-35bc1f6a7093 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:22.436273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3867-c2b1-4c8f-a484-35bc1f6a7093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:22.438888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc3867-c2b1-4c8f-a484-35bc1f6a7093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:22.439983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc3867-c2b1-4c8f-a484-35bc1f6a7093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:22.441787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc3867-c2b1-4c8f-a484-35bc1f6a7093 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:22.472258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 842/1675] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:22.476862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc793-df2f-4a2f-8d52-2f2736d8789e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:22.492621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc793-df2f-4a2f-8d52-2f2736d8789e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:22.502648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff571f-8726-4514-9dc2-44feab0e56f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:22.505861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff571f-8726-4514-9dc2-44feab0e56f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:22.510851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff571f-8726-4514-9dc2-44feab0e56f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:22.511489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff571f-8726-4514-9dc2-44feab0e56f9 admin admin] PATH_INFO: `/v3/projects/8437cc6af4ed477f88dbea8d1ec38297/users/8bc15f227e2d48289bc0f30210291993/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:22.513962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff571f-8726-4514-9dc2-44feab0e56f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8437cc6af4ed477f88dbea8d1ec38297, user_id=8bc15f227e2d48289bc0f30210291993, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:22.517291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff571f-8726-4514-9dc2-44feab0e56f9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:22.549897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 834/1676] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:22 2022] PUT /identity/v3/projects/8437cc6af4ed477f88dbea8d1ec38297/users/8bc15f227e2d48289bc0f30210291993/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:22.563831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383884f8-ad27-4189-a5c1-a2fa1d47e642 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:22.564256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383884f8-ad27-4189-a5c1-a2fa1d47e642 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:22.564621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383884f8-ad27-4189-a5c1-a2fa1d47e642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:22.629391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-383884f8-ad27-4189-a5c1-a2fa1d47e642 None None] MFA Rules not processed for user `8bc15f227e2d48289bc0f30210291993`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:22.631921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-383884f8-ad27-4189-a5c1-a2fa1d47e642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:22.733098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 843/1677] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:22.751972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:22.760862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff571f-8726-4514-9dc2-44feab0e56f9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:22.762916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-08ff571f-8726-4514-9dc2-44feab0e56f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:22.956952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff571f-8726-4514-9dc2-44feab0e56f9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:22.998308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503d0ff-96a1-4c1b-907f-dc7d44c0d0cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:23.001568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503d0ff-96a1-4c1b-907f-dc7d44c0d0cc service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:23.002740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503d0ff-96a1-4c1b-907f-dc7d44c0d0cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:23.005155 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503d0ff-96a1-4c1b-907f-dc7d44c0d0cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:23.019633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503d0ff-96a1-4c1b-907f-dc7d44c0d0cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:23.021196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503d0ff-96a1-4c1b-907f-dc7d44c0d0cc service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:23.034639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 835/1678] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:22 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:23.455609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383884f8-ad27-4189-a5c1-a2fa1d47e642 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:23.477317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383884f8-ad27-4189-a5c1-a2fa1d47e642 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:23.480480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec33001-866f-411d-96f6-d8c04b7f7076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:23.482302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec33001-866f-411d-96f6-d8c04b7f7076 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:23.484153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec33001-866f-411d-96f6-d8c04b7f7076 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:23.485589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec33001-866f-411d-96f6-d8c04b7f7076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:23.615158 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec33001-866f-411d-96f6-d8c04b7f7076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:23.617170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec33001-866f-411d-96f6-d8c04b7f7076 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:23.646117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 844/1679] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:24.736867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:26.345011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c2061-db48-4edf-98c6-06d1dfe85b9b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:26.345994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c2061-db48-4edf-98c6-06d1dfe85b9b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:26.345994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c2061-db48-4edf-98c6-06d1dfe85b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:26.421262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b69c2061-db48-4edf-98c6-06d1dfe85b9b None None] MFA Rules not processed for user `5c35bd3640694b0dbce13c713fd68306`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:26.422799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b69c2061-db48-4edf-98c6-06d1dfe85b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:26.655550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:26.675366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 836/1680] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:26.689140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5dc98-9beb-4bed-8641-dbbe5db52779 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:26.689681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5dc98-9beb-4bed-8641-dbbe5db52779 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:26.689801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5dc98-9beb-4bed-8641-dbbe5db52779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:26.793752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-aaa5dc98-9beb-4bed-8641-dbbe5db52779 None None] MFA Rules not processed for user `5c35bd3640694b0dbce13c713fd68306`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:26.802673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-aaa5dc98-9beb-4bed-8641-dbbe5db52779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:26.835231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 845/1681] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:26.851571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c2061-db48-4edf-98c6-06d1dfe85b9b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:26.855496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b69c2061-db48-4edf-98c6-06d1dfe85b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:26.997652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af600b90-8db2-4ef9-bb70-80ecf936f10e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:26.998671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af600b90-8db2-4ef9-bb70-80ecf936f10e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:26.999381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af600b90-8db2-4ef9-bb70-80ecf936f10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:27.038694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-af600b90-8db2-4ef9-bb70-80ecf936f10e None None] MFA Rules not processed for user `707a899d6c5d4c4ea1c58e0de6a85b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:27.043000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-af600b90-8db2-4ef9-bb70-80ecf936f10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:27.147869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c2061-db48-4edf-98c6-06d1dfe85b9b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:27.158351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9bee3-be67-40ab-b49b-6c706cbf2951 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:27.159804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9bee3-be67-40ab-b49b-6c706cbf2951 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:27.159804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9bee3-be67-40ab-b49b-6c706cbf2951 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:27.159804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9bee3-be67-40ab-b49b-6c706cbf2951 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:27.189696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9bee3-be67-40ab-b49b-6c706cbf2951 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:27.191059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9bee3-be67-40ab-b49b-6c706cbf2951 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:27.238944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 846/1682] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:27.239400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 837/1683] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:27.317197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a314-ec3a-4298-994f-b14228775467 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:27.317498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a314-ec3a-4298-994f-b14228775467 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:27.317782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283a314-ec3a-4298-994f-b14228775467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:27.366082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0283a314-ec3a-4298-994f-b14228775467 None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:27.368353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0283a314-ec3a-4298-994f-b14228775467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:27.549886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 847/1684] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:24:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:28.244416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9bee3-be67-40ab-b49b-6c706cbf2951 service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:28.249485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0dd9bee3-be67-40ab-b49b-6c706cbf2951 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:28.454794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9bee3-be67-40ab-b49b-6c706cbf2951 service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:28.462128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa34bf-fa88-4cda-b306-e64a9ee1f220 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:28.463457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa34bf-fa88-4cda-b306-e64a9ee1f220 service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:28.463953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa34bf-fa88-4cda-b306-e64a9ee1f220 service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:28.464454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa34bf-fa88-4cda-b306-e64a9ee1f220 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:28.475253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa34bf-fa88-4cda-b306-e64a9ee1f220 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:28.478249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa34bf-fa88-4cda-b306-e64a9ee1f220 service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:28.501619 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 838/1685] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:24:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:24:28.506040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf5327-6e21-443e-b250-9872738e8eaf None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:28.508653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf5327-6e21-443e-b250-9872738e8eaf None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:28.510276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf5327-6e21-443e-b250-9872738e8eaf None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:28.517049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 848/1686] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 19 12:24:28 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:24:28.525319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd59fb8-7c1f-4846-a705-197b653de13e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:28.525931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd59fb8-7c1f-4846-a705-197b653de13e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:28.526378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd59fb8-7c1f-4846-a705-197b653de13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:28.567326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9cd59fb8-7c1f-4846-a705-197b653de13e None None] MFA Rules not processed for user `8b266ea9964c4a179c2743b9cf6478eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:28.567326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9cd59fb8-7c1f-4846-a705-197b653de13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:28.763562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 839/1687] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:24:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:28.785063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf5327-6e21-443e-b250-9872738e8eaf None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:28.794930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-82bf5327-6e21-443e-b250-9872738e8eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:28.831090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf5327-6e21-443e-b250-9872738e8eaf None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:28.836590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86130b-b530-450e-abeb-21ef45957d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:28.837697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86130b-b530-450e-abeb-21ef45957d3f service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:28.838202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86130b-b530-450e-abeb-21ef45957d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:28.838730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86130b-b530-450e-abeb-21ef45957d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:28.865153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86130b-b530-450e-abeb-21ef45957d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:28.867643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86130b-b530-450e-abeb-21ef45957d3f service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:28.891282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 849/1688] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:29.849300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cb393-56fc-4d62-88a5-3713317539aa None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:29.850575 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cb393-56fc-4d62-88a5-3713317539aa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:29.850879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cb393-56fc-4d62-88a5-3713317539aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:29.971327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1d5cb393-56fc-4d62-88a5-3713317539aa None None] MFA Rules not processed for user `f0a98f1c068843a684f6765d41cc4e49`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:29.974999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1d5cb393-56fc-4d62-88a5-3713317539aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:30.007456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:30.266616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 840/1689] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:31.284162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86130b-b530-450e-abeb-21ef45957d3f service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:31.322879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86130b-b530-450e-abeb-21ef45957d3f service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:31.340428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6868f-68ad-49dc-8942-385484e3a149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:31.340428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6868f-68ad-49dc-8942-385484e3a149 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:31.340428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6868f-68ad-49dc-8942-385484e3a149 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:31.340428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6868f-68ad-49dc-8942-385484e3a149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:31.411232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6868f-68ad-49dc-8942-385484e3a149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:31.425915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6868f-68ad-49dc-8942-385484e3a149 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:31.482972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 850/1690] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:31.546359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723b309-804d-4c8b-919c-a0963c246e67 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:31.546359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723b309-804d-4c8b-919c-a0963c246e67 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:31.546359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723b309-804d-4c8b-919c-a0963c246e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:31.628498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d6cc7-b3ff-440d-ab63-badcd23ee8c3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:31.629074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d6cc7-b3ff-440d-ab63-badcd23ee8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:31.629693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d6cc7-b3ff-440d-ab63-badcd23ee8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:31.665725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c68d6cc7-b3ff-440d-ab63-badcd23ee8c3 None None] MFA Rules not processed for user `8b266ea9964c4a179c2743b9cf6478eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:31.670315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c68d6cc7-b3ff-440d-ab63-badcd23ee8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:31.682929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a723b309-804d-4c8b-919c-a0963c246e67 None None] MFA Rules not processed for user `746a70534fe5419ebe6d5e97a7a041f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:31.685785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a723b309-804d-4c8b-919c-a0963c246e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:31.686310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 851/1691] 10.222.0.77 () {58 vars in 1015 bytes} [Thu May 19 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:31.701636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d6cc7-b3ff-440d-ab63-badcd23ee8c3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:31.705233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c68d6cc7-b3ff-440d-ab63-badcd23ee8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:31.732225 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d6cc7-b3ff-440d-ab63-badcd23ee8c3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:31.757181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce548eea-7045-4c57-8678-d5a56bd96cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:31.760680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce548eea-7045-4c57-8678-d5a56bd96cad service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:31.763725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce548eea-7045-4c57-8678-d5a56bd96cad service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:31.764422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce548eea-7045-4c57-8678-d5a56bd96cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:31.799091 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce548eea-7045-4c57-8678-d5a56bd96cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:31.800886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce548eea-7045-4c57-8678-d5a56bd96cad service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:31.843024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 852/1692] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:31.999721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 841/1693] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:32.012839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb617-8ab6-43b8-b667-c015abe31dac None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:32.013455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb617-8ab6-43b8-b667-c015abe31dac None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:32.013944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb617-8ab6-43b8-b667-c015abe31dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:32.079711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-483eb617-8ab6-43b8-b667-c015abe31dac None None] MFA Rules not processed for user `5c35bd3640694b0dbce13c713fd68306`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:32.082575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-483eb617-8ab6-43b8-b667-c015abe31dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:32.122912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 853/1694] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:32.153119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723b309-804d-4c8b-919c-a0963c246e67 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:32.182577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723b309-804d-4c8b-919c-a0963c246e67 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:32.211268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecbf23-6cc4-4074-a532-b9ac8b6a35ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:32.212214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecbf23-6cc4-4074-a532-b9ac8b6a35ff service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:32.212611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecbf23-6cc4-4074-a532-b9ac8b6a35ff service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:32.212999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecbf23-6cc4-4074-a532-b9ac8b6a35ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:32.240828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecbf23-6cc4-4074-a532-b9ac8b6a35ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:32.242952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecbf23-6cc4-4074-a532-b9ac8b6a35ff service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:32.286878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 842/1695] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:37.327624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb617-8ab6-43b8-b667-c015abe31dac None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:37.618455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:37.629967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e87e-5545-4b42-97a9-6669e119f9d3 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:37.630571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e87e-5545-4b42-97a9-6669e119f9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:37.630791 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e87e-5545-4b42-97a9-6669e119f9d3 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:37.633560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 843/1696] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:24:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:24:37.673925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a35844-2a46-4acc-9ba6-7d45441b778e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:37.674706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a35844-2a46-4acc-9ba6-7d45441b778e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:37.675176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a35844-2a46-4acc-9ba6-7d45441b778e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:37.756211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb617-8ab6-43b8-b667-c015abe31dac None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:37.766345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c5a35844-2a46-4acc-9ba6-7d45441b778e None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:37.775134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c5a35844-2a46-4acc-9ba6-7d45441b778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:37.784490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:37.785539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:37.785831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:37.786117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] PATH_INFO: `/v3/users/253078990e29460db068751dcab6c8d0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:37.793454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253078990e29460db068751dcab6c8d0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:37.809814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:37.936694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 844/1697] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:38.352551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253078990e29460db068751dcab6c8d0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:38.370503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253078990e29460db068751dcab6c8d0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:38.372332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253078990e29460db068751dcab6c8d0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:38.383353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253078990e29460db068751dcab6c8d0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:38.482691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253078990e29460db068751dcab6c8d0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:38.553014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 854/1698] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:37 2022] DELETE /identity/v3/users/253078990e29460db068751dcab6c8d0 => generated 0 bytes in 1227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:38.558671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bf635-0388-4df4-8788-49f082f75daa None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:38.559156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bf635-0388-4df4-8788-49f082f75daa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:38.559647 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3bf635-0388-4df4-8788-49f082f75daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:38.589782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ef3bf635-0388-4df4-8788-49f082f75daa None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:38.594258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ef3bf635-0388-4df4-8788-49f082f75daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:38.866010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 845/1699] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:38.933675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:38.935299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:38.974286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bf635-0388-4df4-8788-49f082f75daa None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:38.976164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ef3bf635-0388-4df4-8788-49f082f75daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:39.080084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:39.162662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb3177-a940-4c3c-bcab-1fefc255b045 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:39.192132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0cad5-f1e0-450b-84d8-4733bff1129c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:39.192132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0cad5-f1e0-450b-84d8-4733bff1129c service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:39.192132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0cad5-f1e0-450b-84d8-4733bff1129c service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:39.192132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0cad5-f1e0-450b-84d8-4733bff1129c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:39.218019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0cad5-f1e0-450b-84d8-4733bff1129c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:39.243936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0cad5-f1e0-450b-84d8-4733bff1129c service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:39.320188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3bf635-0388-4df4-8788-49f082f75daa None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:39.334601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 855/1700] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:39.338899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0cad5-f1e0-450b-84d8-4733bff1129c service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:39.341210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7301df9-8c9c-46f6-9a76-63131ece248a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:39.342379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7301df9-8c9c-46f6-9a76-63131ece248a service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:39.342942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7301df9-8c9c-46f6-9a76-63131ece248a service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:39.344046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7301df9-8c9c-46f6-9a76-63131ece248a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:39.345580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-07a0cad5-f1e0-450b-84d8-4733bff1129c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:39.356914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c7301df9-8c9c-46f6-9a76-63131ece248a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:39.387795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0cad5-f1e0-450b-84d8-4733bff1129c service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:39.398940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:39.401847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:39.401847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:39.401847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] PATH_INFO: `/v3/users/8fe038c184634d46b45c52ab9d23eb4d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:39.402086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7301df9-8c9c-46f6-9a76-63131ece248a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:39.405343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7301df9-8c9c-46f6-9a76-63131ece248a service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:39.413181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fe038c184634d46b45c52ab9d23eb4d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:39.423925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:39.443151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 846/1701] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:39.704577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:39.728481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fe038c184634d46b45c52ab9d23eb4d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:39.746184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fe038c184634d46b45c52ab9d23eb4d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:39.753460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fe038c184634d46b45c52ab9d23eb4d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:39.755518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fe038c184634d46b45c52ab9d23eb4d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:39.872105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fe038c184634d46b45c52ab9d23eb4d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:39.944003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 856/1702] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:39 2022] DELETE /identity/v3/users/8fe038c184634d46b45c52ab9d23eb4d => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:39.949796 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb5c19-ca04-4691-860d-1e761e7be5de None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:39.951212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb5c19-ca04-4691-860d-1e761e7be5de None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:39.951919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb5c19-ca04-4691-860d-1e761e7be5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:40.039758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-37cb5c19-ca04-4691-860d-1e761e7be5de None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:40.050265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-37cb5c19-ca04-4691-860d-1e761e7be5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:40.256226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 847/1703] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:40.534851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:40.536320 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:40.788955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977eb7-4e53-466d-9738-3e60ffcbf1e8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:40.802872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac703fe6-059b-404f-aef0-f443cf1f51dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:40.803560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac703fe6-059b-404f-aef0-f443cf1f51dc service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:40.803847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac703fe6-059b-404f-aef0-f443cf1f51dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:40.804124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac703fe6-059b-404f-aef0-f443cf1f51dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:40.837434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac703fe6-059b-404f-aef0-f443cf1f51dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:40.839001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac703fe6-059b-404f-aef0-f443cf1f51dc service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:40.889257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 857/1704] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:24:42.397369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb5c19-ca04-4691-860d-1e761e7be5de None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:42.397369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-37cb5c19-ca04-4691-860d-1e761e7be5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:42.442294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb5c19-ca04-4691-860d-1e761e7be5de None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:42.470270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:42.471842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:42.472846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:42.474507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] PATH_INFO: `/v3/projects/5258d7da4efe4da78362ec8d6b059a39` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:42.478636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=5258d7da4efe4da78362ec8d6b059a39)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:42.481851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:42.764063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5258d7da4efe4da78362ec8d6b059a39', 'request_id': 'req-2f9755f2-712b-44a5-99a1-372df03cecfe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:42.784161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5258d7da4efe4da78362ec8d6b059a39', 'request_id': 'req-2f9755f2-712b-44a5-99a1-372df03cecfe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:42.890802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5258d7da4efe4da78362ec8d6b059a39', 'request_id': 'req-2f9755f2-712b-44a5-99a1-372df03cecfe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:42.955455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] The token cache is being invalidate because project 5258d7da4efe4da78362ec8d6b059a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:24:42.955697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:42.959710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2f9755f2-712b-44a5-99a1-372df03cecfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:42.962087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 848/1705] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:24:42 2022] DELETE /identity/v3/projects/5258d7da4efe4da78362ec8d6b059a39 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:42.966395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac703fe6-059b-404f-aef0-f443cf1f51dc service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:42.984834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac703fe6-059b-404f-aef0-f443cf1f51dc service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:43.279685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:43.281606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:43.282315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:43.283284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] PATH_INFO: `/v3/users/8bc15f227e2d48289bc0f30210291993` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:43.290893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc15f227e2d48289bc0f30210291993)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:43.307900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:43.790527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc15f227e2d48289bc0f30210291993'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:43.816222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc15f227e2d48289bc0f30210291993'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:43.817673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc15f227e2d48289bc0f30210291993'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:43.819038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc15f227e2d48289bc0f30210291993'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:43.908695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc15f227e2d48289bc0f30210291993'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:43.910698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:43.940844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 858/1706] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:42 2022] DELETE /identity/v3/users/8bc15f227e2d48289bc0f30210291993 => generated 0 bytes in 968 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:44.130194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ff999-14d7-41ad-989b-e29bc6d17b10 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:44.133804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ff999-14d7-41ad-989b-e29bc6d17b10 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:44.133804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ff999-14d7-41ad-989b-e29bc6d17b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:44.185000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b43ff999-14d7-41ad-989b-e29bc6d17b10 None None] MFA Rules not processed for user `f0a98f1c068843a684f6765d41cc4e49`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:44.192105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b43ff999-14d7-41ad-989b-e29bc6d17b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:44.255066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:44.340092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 849/1707] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:44.527628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:44.529718 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:44.723663 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70d4d4-713d-4b02-ab37-4dbbe3c32d6a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:44.750901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:44.768837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:44.769138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:44.769425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] PATH_INFO: `/v3/projects/a332bbbc8cd04a7ab5b13aa33ef7a783` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:44.774242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a332bbbc8cd04a7ab5b13aa33ef7a783)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:44.777323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:45.153083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a332bbbc8cd04a7ab5b13aa33ef7a783', 'request_id': 'req-97aa8865-bf36-499b-8052-d4b251c4a6f7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:45.171894 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a332bbbc8cd04a7ab5b13aa33ef7a783', 'request_id': 'req-97aa8865-bf36-499b-8052-d4b251c4a6f7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:45.357383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a332bbbc8cd04a7ab5b13aa33ef7a783', 'request_id': 'req-97aa8865-bf36-499b-8052-d4b251c4a6f7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:45.382505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] The token cache is being invalidate because project a332bbbc8cd04a7ab5b13aa33ef7a783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:24:45.382987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:45.384260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-97aa8865-bf36-499b-8052-d4b251c4a6f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:45.386555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 859/1708] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:24:44 2022] DELETE /identity/v3/projects/a332bbbc8cd04a7ab5b13aa33ef7a783 => generated 0 bytes in 859 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:45.393444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa614b-5e50-4ebc-a044-37ec8ca0fcad None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:45.394216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa614b-5e50-4ebc-a044-37ec8ca0fcad None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:45.394598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa614b-5e50-4ebc-a044-37ec8ca0fcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:45.451584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-06aa614b-5e50-4ebc-a044-37ec8ca0fcad None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:45.452934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-06aa614b-5e50-4ebc-a044-37ec8ca0fcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:45.664979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 850/1709] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:45.675941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ec21c-bb98-40df-afc4-00930a44c944 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:45.676553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ec21c-bb98-40df-afc4-00930a44c944 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:45.676553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ec21c-bb98-40df-afc4-00930a44c944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:45.720885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-919ec21c-bb98-40df-afc4-00930a44c944 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:45.722421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-919ec21c-bb98-40df-afc4-00930a44c944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:45.734441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 860/1710] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:45.745261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa614b-5e50-4ebc-a044-37ec8ca0fcad None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:45.768377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa614b-5e50-4ebc-a044-37ec8ca0fcad None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:45.773758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4603347-6aad-46e8-8604-72f125127498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:45.774201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4603347-6aad-46e8-8604-72f125127498 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:45.774790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4603347-6aad-46e8-8604-72f125127498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:45.775250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4603347-6aad-46e8-8604-72f125127498 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:45.776185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4603347-6aad-46e8-8604-72f125127498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:45.776666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4603347-6aad-46e8-8604-72f125127498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:45.778571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4603347-6aad-46e8-8604-72f125127498 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:45.826009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 851/1711] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:45.826852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ec21c-bb98-40df-afc4-00930a44c944 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:45.832621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ec21c-bb98-40df-afc4-00930a44c944 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:45.845950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc45048-acb3-4a59-bc9d-aef8c52e5836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:45.845950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc45048-acb3-4a59-bc9d-aef8c52e5836 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:45.845950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc45048-acb3-4a59-bc9d-aef8c52e5836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:45.845950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc45048-acb3-4a59-bc9d-aef8c52e5836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:45.846794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc45048-acb3-4a59-bc9d-aef8c52e5836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:45.866837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc45048-acb3-4a59-bc9d-aef8c52e5836 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:47.162297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 861/1712] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:45 2022] POST /identity/v3/projects => generated 383 bytes in 1342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:24:47.167116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4603347-6aad-46e8-8604-72f125127498 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:47.178174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4603347-6aad-46e8-8604-72f125127498 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:47.224654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc45048-acb3-4a59-bc9d-aef8c52e5836 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:47.226838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dcc45048-acb3-4a59-bc9d-aef8c52e5836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:47.402733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc45048-acb3-4a59-bc9d-aef8c52e5836 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:47.419049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:47.420160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:47.420448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:47.420721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] PATH_INFO: `/v3/projects/8437cc6af4ed477f88dbea8d1ec38297` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:47.423611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8437cc6af4ed477f88dbea8d1ec38297)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:47.428569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:47.603247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f5e49-ae3a-405e-bd0b-a28bbfc27e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:47.606272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f5e49-ae3a-405e-bd0b-a28bbfc27e86 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:47.606272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f5e49-ae3a-405e-bd0b-a28bbfc27e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:47.606272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f5e49-ae3a-405e-bd0b-a28bbfc27e86 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:47.606272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f5e49-ae3a-405e-bd0b-a28bbfc27e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:47.608360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f5e49-ae3a-405e-bd0b-a28bbfc27e86 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:47.715622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8437cc6af4ed477f88dbea8d1ec38297', 'request_id': 'req-59514e82-03e0-4a13-b817-8cd0a030bae0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:47.733051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8437cc6af4ed477f88dbea8d1ec38297', 'request_id': 'req-59514e82-03e0-4a13-b817-8cd0a030bae0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:47.843859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 852/1713] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:47 2022] POST /identity/v3/users => generated 292 bytes in 677 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:47.846172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df388a-416a-4c3a-b3d9-734f308a80a8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:47.846593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df388a-416a-4c3a-b3d9-734f308a80a8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:47.847147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df388a-416a-4c3a-b3d9-734f308a80a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:47.868282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-64df388a-416a-4c3a-b3d9-734f308a80a8 None None] MFA Rules not processed for user `746a70534fe5419ebe6d5e97a7a041f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:47.869487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-64df388a-416a-4c3a-b3d9-734f308a80a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:47.878060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8437cc6af4ed477f88dbea8d1ec38297', 'request_id': 'req-59514e82-03e0-4a13-b817-8cd0a030bae0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:47.900674 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] The token cache is being invalidate because project 8437cc6af4ed477f88dbea8d1ec38297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:24:47.901302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:47.902085 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:47.908318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 862/1714] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:24:47 2022] DELETE /identity/v3/projects/8437cc6af4ed477f88dbea8d1ec38297 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:47.909676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:47.911498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:47.963218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 853/1715] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:47.972278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b677e-abb0-49aa-a2f0-d63e122d9ca3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:47.972278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b677e-abb0-49aa-a2f0-d63e122d9ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:47.972278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b677e-abb0-49aa-a2f0-d63e122d9ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:47.996692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b36b677e-abb0-49aa-a2f0-d63e122d9ca3 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:47.998122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b36b677e-abb0-49aa-a2f0-d63e122d9ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:48.127257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 854/1716] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:48.135224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26685f5a-697b-4a1b-ae9f-d77aaaf18c31 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.135629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26685f5a-697b-4a1b-ae9f-d77aaaf18c31 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.136178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26685f5a-697b-4a1b-ae9f-d77aaaf18c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.167804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59514e82-03e0-4a13-b817-8cd0a030bae0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:48.186070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-26685f5a-697b-4a1b-ae9f-d77aaaf18c31 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:48.187865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-26685f5a-697b-4a1b-ae9f-d77aaaf18c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:48.191562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f531655-5644-42c9-948c-ac1bb6fcbd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:48.192275 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f531655-5644-42c9-948c-ac1bb6fcbd70 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.192837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f531655-5644-42c9-948c-ac1bb6fcbd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.193391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f531655-5644-42c9-948c-ac1bb6fcbd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.194295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f531655-5644-42c9-948c-ac1bb6fcbd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:48.196205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f531655-5644-42c9-948c-ac1bb6fcbd70 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:48.199146 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 855/1717] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:48.204218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26685f5a-697b-4a1b-ae9f-d77aaaf18c31 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:48.210268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 863/1718] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:48.214130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f531655-5644-42c9-948c-ac1bb6fcbd70 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:48.221849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26685f5a-697b-4a1b-ae9f-d77aaaf18c31 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:48.228367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f531655-5644-42c9-948c-ac1bb6fcbd70 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:48.231104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81b569-ed34-4152-8345-d6cd4ac774a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:48.233845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81b569-ed34-4152-8345-d6cd4ac774a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.234298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81b569-ed34-4152-8345-d6cd4ac774a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.237265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81b569-ed34-4152-8345-d6cd4ac774a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.240808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81b569-ed34-4152-8345-d6cd4ac774a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:48.241147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81b569-ed34-4152-8345-d6cd4ac774a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:48.242867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81b569-ed34-4152-8345-d6cd4ac774a1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:48.274471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 856/1719] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:24:48.280387 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81b569-ed34-4152-8345-d6cd4ac774a1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:48.286335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b14a9-255e-4b4f-bf7a-fbb59e7144ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:48.289520 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b14a9-255e-4b4f-bf7a-fbb59e7144ce admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.289520 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b14a9-255e-4b4f-bf7a-fbb59e7144ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.289520 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b14a9-255e-4b4f-bf7a-fbb59e7144ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.292283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b14a9-255e-4b4f-bf7a-fbb59e7144ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:48.298752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b14a9-255e-4b4f-bf7a-fbb59e7144ce admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:48.301859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81b569-ed34-4152-8345-d6cd4ac774a1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:48.321594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c683e11-deb3-41ad-b884-6d5a42d28cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:48.322509 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 864/1720] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:48.326825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b14a9-255e-4b4f-bf7a-fbb59e7144ce admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:48.327206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c683e11-deb3-41ad-b884-6d5a42d28cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.329969 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c683e11-deb3-41ad-b884-6d5a42d28cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.332481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c683e11-deb3-41ad-b884-6d5a42d28cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.333266 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c683e11-deb3-41ad-b884-6d5a42d28cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:48.340725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b14a9-255e-4b4f-bf7a-fbb59e7144ce admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:48.346833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c683e11-deb3-41ad-b884-6d5a42d28cfc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:48.357874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e0bd5-c6f8-4bdf-8e8b-a3074c2c4204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:48.359790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e0bd5-c6f8-4bdf-8e8b-a3074c2c4204 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.360241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e0bd5-c6f8-4bdf-8e8b-a3074c2c4204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.360582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e0bd5-c6f8-4bdf-8e8b-a3074c2c4204 admin admin] PATH_INFO: `/v3/projects/99df372c1dee4f8b8f70fe5f08494d60/users/c5c04eea555f4567a26422443115eb21/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.418765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e0bd5-c6f8-4bdf-8e8b-a3074c2c4204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99df372c1dee4f8b8f70fe5f08494d60, user_id=c5c04eea555f4567a26422443115eb21, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:48.430084 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e0bd5-c6f8-4bdf-8e8b-a3074c2c4204 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:48.522053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 857/1721] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:48 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:48.527050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa809a3-7616-4fdd-aef1-afe5ab780831 None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.527050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa809a3-7616-4fdd-aef1-afe5ab780831 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.527050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa809a3-7616-4fdd-aef1-afe5ab780831 None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.529308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 858/1722] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:24:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:24:48.531011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa809a3-7616-4fdd-aef1-afe5ab780831 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:48.639073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 865/1723] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:48 2022] PUT /identity/v3/projects/99df372c1dee4f8b8f70fe5f08494d60/users/c5c04eea555f4567a26422443115eb21/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:24:48.644697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72031-4563-49de-8df0-d7ad62bc1381 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.644697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72031-4563-49de-8df0-d7ad62bc1381 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.645425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72031-4563-49de-8df0-d7ad62bc1381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.686290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa809a3-7616-4fdd-aef1-afe5ab780831 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:48.710147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f7f72031-4563-49de-8df0-d7ad62bc1381 None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:48.711877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f7f72031-4563-49de-8df0-d7ad62bc1381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:48.864456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d77ea-c832-4ce0-9dc0-9747f442db29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:48.865151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d77ea-c832-4ce0-9dc0-9747f442db29 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.865440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d77ea-c832-4ce0-9dc0-9747f442db29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.865726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d77ea-c832-4ce0-9dc0-9747f442db29 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.867030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d77ea-c832-4ce0-9dc0-9747f442db29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:48.869023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d77ea-c832-4ce0-9dc0-9747f442db29 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:48.870985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 866/1724] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:24:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:48.872499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c19367-dbfb-4b5f-9b8b-d785dd8eae31 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:48.872861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c19367-dbfb-4b5f-9b8b-d785dd8eae31 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:48.873208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c19367-dbfb-4b5f-9b8b-d785dd8eae31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:48.912581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-28c19367-dbfb-4b5f-9b8b-d785dd8eae31 None None] MFA Rules not processed for user `c5c04eea555f4567a26422443115eb21`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:48.913935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-28c19367-dbfb-4b5f-9b8b-d785dd8eae31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:49.007877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 867/1725] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:49.013295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c19367-dbfb-4b5f-9b8b-d785dd8eae31 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.020091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 859/1726] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:48 2022] POST /identity/v3/users => generated 293 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:49.026117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d77ea-c832-4ce0-9dc0-9747f442db29 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.042847 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d77ea-c832-4ce0-9dc0-9747f442db29 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.050903 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93c22a-2693-4fb5-9c05-00245a87aa52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.051774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93c22a-2693-4fb5-9c05-00245a87aa52 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.052115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93c22a-2693-4fb5-9c05-00245a87aa52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.052458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93c22a-2693-4fb5-9c05-00245a87aa52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.052635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c19367-dbfb-4b5f-9b8b-d785dd8eae31 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.053292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93c22a-2693-4fb5-9c05-00245a87aa52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.055068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93c22a-2693-4fb5-9c05-00245a87aa52 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:49.069897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead2ecf-23a2-4cd2-a32f-d8fa39e65da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.073155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead2ecf-23a2-4cd2-a32f-d8fa39e65da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.074280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead2ecf-23a2-4cd2-a32f-d8fa39e65da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.075040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead2ecf-23a2-4cd2-a32f-d8fa39e65da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.076409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead2ecf-23a2-4cd2-a32f-d8fa39e65da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.083419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead2ecf-23a2-4cd2-a32f-d8fa39e65da3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:49.096795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 860/1727] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:49.104543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93c22a-2693-4fb5-9c05-00245a87aa52 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.129449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93c22a-2693-4fb5-9c05-00245a87aa52 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.144540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189e3a4-5690-461e-8802-f5ddc0a12313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.146563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189e3a4-5690-461e-8802-f5ddc0a12313 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.147499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189e3a4-5690-461e-8802-f5ddc0a12313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.148247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189e3a4-5690-461e-8802-f5ddc0a12313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.149867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189e3a4-5690-461e-8802-f5ddc0a12313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.154884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189e3a4-5690-461e-8802-f5ddc0a12313 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:49.176250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 861/1728] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:49.179563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189e3a4-5690-461e-8802-f5ddc0a12313 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.196976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189e3a4-5690-461e-8802-f5ddc0a12313 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.212393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae235f-bbaf-47cb-bb80-cdbc0449b750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.212393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae235f-bbaf-47cb-bb80-cdbc0449b750 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.212393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae235f-bbaf-47cb-bb80-cdbc0449b750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.212393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae235f-bbaf-47cb-bb80-cdbc0449b750 admin admin] PATH_INFO: `/v3/projects/5cbe0dc7a4a84d6eaa60215d855dcefe/users/108dff16aae44d82880e8e6c4afe90cb/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.230614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae235f-bbaf-47cb-bb80-cdbc0449b750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbe0dc7a4a84d6eaa60215d855dcefe, user_id=108dff16aae44d82880e8e6c4afe90cb, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.233285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae235f-bbaf-47cb-bb80-cdbc0449b750 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:49.251652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 868/1729] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:49 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:49.253248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead2ecf-23a2-4cd2-a32f-d8fa39e65da3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.255182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-aead2ecf-23a2-4cd2-a32f-d8fa39e65da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:49.342023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 862/1730] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:49 2022] PUT /identity/v3/projects/5cbe0dc7a4a84d6eaa60215d855dcefe/users/108dff16aae44d82880e8e6c4afe90cb/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:49.345925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae235f-bbaf-47cb-bb80-cdbc0449b750 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.395156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead2ecf-23a2-4cd2-a32f-d8fa39e65da3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.466018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae235f-bbaf-47cb-bb80-cdbc0449b750 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.475102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875a621-d7c1-4700-8615-2824291efb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.476250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875a621-d7c1-4700-8615-2824291efb53 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.476753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875a621-d7c1-4700-8615-2824291efb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.477258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875a621-d7c1-4700-8615-2824291efb53 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.478500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875a621-d7c1-4700-8615-2824291efb53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.482246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875a621-d7c1-4700-8615-2824291efb53 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:49.488587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f11fd-dce3-499b-be55-acf10d6235df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.489548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f11fd-dce3-499b-be55-acf10d6235df service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.490600 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f11fd-dce3-499b-be55-acf10d6235df service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.490600 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f11fd-dce3-499b-be55-acf10d6235df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.501190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f11fd-dce3-499b-be55-acf10d6235df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.502883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f11fd-dce3-499b-be55-acf10d6235df service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:49.515115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 869/1731] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:49.516498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6343cc-8724-403a-b3b3-76ffd9c3544a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.516788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6343cc-8724-403a-b3b3-76ffd9c3544a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.517089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6343cc-8724-403a-b3b3-76ffd9c3544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.553511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1d6343cc-8724-403a-b3b3-76ffd9c3544a None None] MFA Rules not processed for user `108dff16aae44d82880e8e6c4afe90cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:49.555789 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1d6343cc-8724-403a-b3b3-76ffd9c3544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:49.722699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 870/1732] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:49.722699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6343cc-8724-403a-b3b3-76ffd9c3544a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.729254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6343cc-8724-403a-b3b3-76ffd9c3544a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.739868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36480bc7-40f7-4018-b425-2e74232ebc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.740563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36480bc7-40f7-4018-b425-2e74232ebc3e admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.740850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36480bc7-40f7-4018-b425-2e74232ebc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.741132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36480bc7-40f7-4018-b425-2e74232ebc3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.741839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36480bc7-40f7-4018-b425-2e74232ebc3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.744538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36480bc7-40f7-4018-b425-2e74232ebc3e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:49.772768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 863/1733] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:49 2022] POST /identity/v3/users => generated 297 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:49.776864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875a621-d7c1-4700-8615-2824291efb53 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.807128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875a621-d7c1-4700-8615-2824291efb53 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.821951 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89100954-32bb-41c4-add3-4cdefd62270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.822448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89100954-32bb-41c4-add3-4cdefd62270c admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.822645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89100954-32bb-41c4-add3-4cdefd62270c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.825094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89100954-32bb-41c4-add3-4cdefd62270c admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.826114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89100954-32bb-41c4-add3-4cdefd62270c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.828093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89100954-32bb-41c4-add3-4cdefd62270c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:49.853061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 864/1734] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:49.857449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89100954-32bb-41c4-add3-4cdefd62270c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.871069 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89100954-32bb-41c4-add3-4cdefd62270c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.880708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dca08b-6281-404f-b835-21bdfc9a3f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.884908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dca08b-6281-404f-b835-21bdfc9a3f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.884908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dca08b-6281-404f-b835-21bdfc9a3f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.884908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dca08b-6281-404f-b835-21bdfc9a3f9f admin admin] PATH_INFO: `/v3/projects/f6e8ade56f0b449cbdbbdaac0daff03e/users/60e5b0fc11bf4982aae936e36a36ddb8/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.889134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 871/1735] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:49 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:49.893763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36480bc7-40f7-4018-b425-2e74232ebc3e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:49.907851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dca08b-6281-404f-b835-21bdfc9a3f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e8ade56f0b449cbdbbdaac0daff03e, user_id=60e5b0fc11bf4982aae936e36a36ddb8, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.920416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dca08b-6281-404f-b835-21bdfc9a3f9f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:49.978326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36480bc7-40f7-4018-b425-2e74232ebc3e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:49.983202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c4f5c-3b8c-4126-b7b3-f00457996d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:49.984478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c4f5c-3b8c-4126-b7b3-f00457996d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:49.984765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c4f5c-3b8c-4126-b7b3-f00457996d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:49.989561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c4f5c-3b8c-4126-b7b3-f00457996d12 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:49.989561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c4f5c-3b8c-4126-b7b3-f00457996d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:49.989561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c4f5c-3b8c-4126-b7b3-f00457996d12 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.082333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 865/1736] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:49 2022] PUT /identity/v3/projects/f6e8ade56f0b449cbdbbdaac0daff03e/users/60e5b0fc11bf4982aae936e36a36ddb8/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:50.101733 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dca08b-6281-404f-b835-21bdfc9a3f9f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:50.162501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 872/1737] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:49 2022] POST /identity/v3/users => generated 300 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:24:50.164692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c4f5c-3b8c-4126-b7b3-f00457996d12 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:50.179243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c4f5c-3b8c-4126-b7b3-f00457996d12 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:50.182110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dca08b-6281-404f-b835-21bdfc9a3f9f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:50.185713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb99e4-ff1d-4fa5-915c-57e2d9b2b7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:50.187201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb99e4-ff1d-4fa5-915c-57e2d9b2b7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.188285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb99e4-ff1d-4fa5-915c-57e2d9b2b7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.188508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb99e4-ff1d-4fa5-915c-57e2d9b2b7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.189264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb99e4-ff1d-4fa5-915c-57e2d9b2b7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:50.189570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd1929-d4bc-40bb-9dd1-dde48a30b0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:50.190256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd1929-d4bc-40bb-9dd1-dde48a30b0ba admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.190796 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd1929-d4bc-40bb-9dd1-dde48a30b0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.190920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb99e4-ff1d-4fa5-915c-57e2d9b2b7b7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.190993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd1929-d4bc-40bb-9dd1-dde48a30b0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.191715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd1929-d4bc-40bb-9dd1-dde48a30b0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:50.193718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd1929-d4bc-40bb-9dd1-dde48a30b0ba admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.203478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 873/1738] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:24:50.205560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 866/1739] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:50.207838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd1929-d4bc-40bb-9dd1-dde48a30b0ba admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:50.222487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb99e4-ff1d-4fa5-915c-57e2d9b2b7b7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:50.222818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd1929-d4bc-40bb-9dd1-dde48a30b0ba admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:50.228021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb100b5f-5acd-4eb3-9e04-7695ac2dc5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:50.229172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb100b5f-5acd-4eb3-9e04-7695ac2dc5bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.229457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb100b5f-5acd-4eb3-9e04-7695ac2dc5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.230190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb100b5f-5acd-4eb3-9e04-7695ac2dc5bf admin admin] PATH_INFO: `/v3/projects/b1c7c25d175a4fd8aa1f2034b9e6d6da/users/9cc7585a11fc4ce094aeb986e7d6e4fd/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.253991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb99e4-ff1d-4fa5-915c-57e2d9b2b7b7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:50.253991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a8f96-26e6-48a0-9bf5-33249e431e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:50.253991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a8f96-26e6-48a0-9bf5-33249e431e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.253991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a8f96-26e6-48a0-9bf5-33249e431e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.255055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a8f96-26e6-48a0-9bf5-33249e431e25 admin admin] PATH_INFO: `/v3/projects/f6e8ade56f0b449cbdbbdaac0daff03e/users/60e5b0fc11bf4982aae936e36a36ddb8/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.255055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a8f96-26e6-48a0-9bf5-33249e431e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e8ade56f0b449cbdbbdaac0daff03e, user_id=60e5b0fc11bf4982aae936e36a36ddb8, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:50.258750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a8f96-26e6-48a0-9bf5-33249e431e25 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.262511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb100b5f-5acd-4eb3-9e04-7695ac2dc5bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c7c25d175a4fd8aa1f2034b9e6d6da, user_id=9cc7585a11fc4ce094aeb986e7d6e4fd, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:50.267106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb100b5f-5acd-4eb3-9e04-7695ac2dc5bf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.275649 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 874/1740] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:50 2022] PUT /identity/v3/projects/f6e8ade56f0b449cbdbbdaac0daff03e/users/60e5b0fc11bf4982aae936e36a36ddb8/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:50.286546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d43c5e-c407-4a34-9a29-09942c74b1fe None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.290657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d43c5e-c407-4a34-9a29-09942c74b1fe None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.290657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d43c5e-c407-4a34-9a29-09942c74b1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.317435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-28d43c5e-c407-4a34-9a29-09942c74b1fe None None] MFA Rules not processed for user `60e5b0fc11bf4982aae936e36a36ddb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:50.318638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-28d43c5e-c407-4a34-9a29-09942c74b1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:50.398450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 875/1741] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:50.398936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 867/1742] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:50 2022] PUT /identity/v3/projects/b1c7c25d175a4fd8aa1f2034b9e6d6da/users/9cc7585a11fc4ce094aeb986e7d6e4fd/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:24:50.406135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d43c5e-c407-4a34-9a29-09942c74b1fe None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:50.406442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb100b5f-5acd-4eb3-9e04-7695ac2dc5bf admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:50.408135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-bb100b5f-5acd-4eb3-9e04-7695ac2dc5bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:50.477871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb100b5f-5acd-4eb3-9e04-7695ac2dc5bf admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:50.479612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d43c5e-c407-4a34-9a29-09942c74b1fe None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:50.482662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201016a-ddae-441d-ac22-29952ad68238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:50.483322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201016a-ddae-441d-ac22-29952ad68238 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.483664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201016a-ddae-441d-ac22-29952ad68238 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.484443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201016a-ddae-441d-ac22-29952ad68238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.542011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4f6fc-433e-4a7e-98f8-d45039411274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:50.542732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4f6fc-433e-4a7e-98f8-d45039411274 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.543026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4f6fc-433e-4a7e-98f8-d45039411274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.543436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4f6fc-433e-4a7e-98f8-d45039411274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.544213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4f6fc-433e-4a7e-98f8-d45039411274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:50.551226 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4f6fc-433e-4a7e-98f8-d45039411274 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.553051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201016a-ddae-441d-ac22-29952ad68238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:50.554525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201016a-ddae-441d-ac22-29952ad68238 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.558183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 876/1743] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:50.562836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4f6fc-433e-4a7e-98f8-d45039411274 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:50.571442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 868/1744] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:24:50.573021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4f6fc-433e-4a7e-98f8-d45039411274 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:50.578781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a8d0c-4cac-4cea-bbfa-e9328fac7751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:50.583034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a8d0c-4cac-4cea-bbfa-e9328fac7751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.583034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a8d0c-4cac-4cea-bbfa-e9328fac7751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.583034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a8d0c-4cac-4cea-bbfa-e9328fac7751 admin admin] PATH_INFO: `/v3/projects/b1c7c25d175a4fd8aa1f2034b9e6d6da/users/9cc7585a11fc4ce094aeb986e7d6e4fd/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.583034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a8d0c-4cac-4cea-bbfa-e9328fac7751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c7c25d175a4fd8aa1f2034b9e6d6da, user_id=9cc7585a11fc4ce094aeb986e7d6e4fd, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:50.584913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a8d0c-4cac-4cea-bbfa-e9328fac7751 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.598551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 877/1745] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:50 2022] PUT /identity/v3/projects/b1c7c25d175a4fd8aa1f2034b9e6d6da/users/9cc7585a11fc4ce094aeb986e7d6e4fd/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:50.608218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e10e4-fd2e-461d-8e4d-c654a8adc5ca None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.608507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e10e4-fd2e-461d-8e4d-c654a8adc5ca None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.608903 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e10e4-fd2e-461d-8e4d-c654a8adc5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.646024 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-735e10e4-fd2e-461d-8e4d-c654a8adc5ca None None] MFA Rules not processed for user `9cc7585a11fc4ce094aeb986e7d6e4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:50.647493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-735e10e4-fd2e-461d-8e4d-c654a8adc5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:50.664827 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:50.697772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a8d0c-4cac-4cea-bbfa-e9328fac7751 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:50.747872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 869/1746] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:50.762074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e10e4-fd2e-461d-8e4d-c654a8adc5ca None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:50.763664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-735e10e4-fd2e-461d-8e4d-c654a8adc5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:50.785604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a8d0c-4cac-4cea-bbfa-e9328fac7751 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:50.789764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599a383-1cbf-4c8c-9da2-d79b68577e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:50.790427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599a383-1cbf-4c8c-9da2-d79b68577e0f service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.790919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599a383-1cbf-4c8c-9da2-d79b68577e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.791586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599a383-1cbf-4c8c-9da2-d79b68577e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.831377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e10e4-fd2e-461d-8e4d-c654a8adc5ca None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:50.835630 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42628090-5914-4228-8ff1-3380a0e7b263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:50.837513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42628090-5914-4228-8ff1-3380a0e7b263 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:50.837513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42628090-5914-4228-8ff1-3380a0e7b263 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:50.837513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42628090-5914-4228-8ff1-3380a0e7b263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:50.848576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42628090-5914-4228-8ff1-3380a0e7b263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:50.850116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42628090-5914-4228-8ff1-3380a0e7b263 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.855473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599a383-1cbf-4c8c-9da2-d79b68577e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:50.861862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599a383-1cbf-4c8c-9da2-d79b68577e0f service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:50.867178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 878/1747] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:24:50.884824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 870/1748] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:24:51.022780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:53.088340 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599a383-1cbf-4c8c-9da2-d79b68577e0f service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:53.162480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599a383-1cbf-4c8c-9da2-d79b68577e0f service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:53.170634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:53.171638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:53.171918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:53.172192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] PATH_INFO: `/v3/users/108dff16aae44d82880e8e6c4afe90cb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:53.173457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=108dff16aae44d82880e8e6c4afe90cb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:53.175697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:53.433748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108dff16aae44d82880e8e6c4afe90cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:53.441258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108dff16aae44d82880e8e6c4afe90cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:53.442146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108dff16aae44d82880e8e6c4afe90cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:53.442832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108dff16aae44d82880e8e6c4afe90cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:53.579307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108dff16aae44d82880e8e6c4afe90cb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:53.603897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 879/1749] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:53 2022] DELETE /identity/v3/users/108dff16aae44d82880e8e6c4afe90cb => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:53.609783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70707d1c-9e9a-41a6-8a3b-8daf3014f653 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:53.611221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70707d1c-9e9a-41a6-8a3b-8daf3014f653 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:53.611221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70707d1c-9e9a-41a6-8a3b-8daf3014f653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:53.630221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-70707d1c-9e9a-41a6-8a3b-8daf3014f653 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:53.631532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-70707d1c-9e9a-41a6-8a3b-8daf3014f653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:53.696718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 871/1750] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:53.707283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:53.709346 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:53.775939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822523-9ce2-4974-b577-8a36f232e69f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:53.779574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a870-cdfe-4eec-aa84-cf590d2ae545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:53.780257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a870-cdfe-4eec-aa84-cf590d2ae545 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:53.781241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a870-cdfe-4eec-aa84-cf590d2ae545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:53.781589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a870-cdfe-4eec-aa84-cf590d2ae545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:53.792160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711a870-cdfe-4eec-aa84-cf590d2ae545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:53.793659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711a870-cdfe-4eec-aa84-cf590d2ae545 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:53.809983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 880/1751] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:54.461105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70707d1c-9e9a-41a6-8a3b-8daf3014f653 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:54.462584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-70707d1c-9e9a-41a6-8a3b-8daf3014f653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:54.477584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70707d1c-9e9a-41a6-8a3b-8daf3014f653 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:54.484602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:54.484937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:54.485232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:54.485512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] PATH_INFO: `/v3/projects/5cbe0dc7a4a84d6eaa60215d855dcefe` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:54.487480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbe0dc7a4a84d6eaa60215d855dcefe)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:54.489902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:54.531873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a870-cdfe-4eec-aa84-cf590d2ae545 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:54.533335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f711a870-cdfe-4eec-aa84-cf590d2ae545 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:54.547198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a870-cdfe-4eec-aa84-cf590d2ae545 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:54.554378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:54.555524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:54.555524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:54.555524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] PATH_INFO: `/v3/users/c5c04eea555f4567a26422443115eb21` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:54.556958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c04eea555f4567a26422443115eb21)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:54.559392 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:54.839451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:54.906893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c04eea555f4567a26422443115eb21'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:54.907028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbe0dc7a4a84d6eaa60215d855dcefe', 'request_id': 'req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:54.909044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbe0dc7a4a84d6eaa60215d855dcefe', 'request_id': 'req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:54.909930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c04eea555f4567a26422443115eb21'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:54.910724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c04eea555f4567a26422443115eb21'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:54.911398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c04eea555f4567a26422443115eb21'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.056766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbe0dc7a4a84d6eaa60215d855dcefe', 'request_id': 'req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.070392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] The token cache is being invalidate because project 5cbe0dc7a4a84d6eaa60215d855dcefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:24:55.070707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.071511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.073637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 872/1752] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:24:54 2022] DELETE /identity/v3/projects/5cbe0dc7a4a84d6eaa60215d855dcefe => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:55.077713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:55.079371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:55.122217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-94a222f7-5b72-47f5-a4ac-5d94c05de69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c04eea555f4567a26422443115eb21'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.147295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 881/1753] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:54 2022] DELETE /identity/v3/users/c5c04eea555f4567a26422443115eb21 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:24:55.151261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd8dde-097a-4dab-a4b5-796a5c5dcda7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:55.154015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf38402-06db-4ccd-8345-d56b5b7055d9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:55.154324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf38402-06db-4ccd-8345-d56b5b7055d9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:55.160302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf38402-06db-4ccd-8345-d56b5b7055d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:55.175535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-dbf38402-06db-4ccd-8345-d56b5b7055d9 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:55.176822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dbf38402-06db-4ccd-8345-d56b5b7055d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:55.247392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:55.249125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:55.250515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:55.251363 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] PATH_INFO: `/v3/users/9cc7585a11fc4ce094aeb986e7d6e4fd` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:55.254086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cc7585a11fc4ce094aeb986e7d6e4fd)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:55.257190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: WARNING py.warnings [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 12:24:55.257190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: warnings.warn(msg) May 19 12:24:55.257190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: May 19 12:24:55.259998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:55.273862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 882/1754] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:55.282269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf38402-06db-4ccd-8345-d56b5b7055d9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:55.284169 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dbf38402-06db-4ccd-8345-d56b5b7055d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:55.350617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf38402-06db-4ccd-8345-d56b5b7055d9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:55.356460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edef0e-6c3a-48a8-9644-4351b4db7c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:55.356460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edef0e-6c3a-48a8-9644-4351b4db7c41 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:55.357779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edef0e-6c3a-48a8-9644-4351b4db7c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:55.357779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edef0e-6c3a-48a8-9644-4351b4db7c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:55.369613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edef0e-6c3a-48a8-9644-4351b4db7c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:55.371581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edef0e-6c3a-48a8-9644-4351b4db7c41 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:55.384846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 883/1755] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:55.480100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cc7585a11fc4ce094aeb986e7d6e4fd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.480754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cc7585a11fc4ce094aeb986e7d6e4fd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.505816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cc7585a11fc4ce094aeb986e7d6e4fd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.506751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cc7585a11fc4ce094aeb986e7d6e4fd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.714278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cc7585a11fc4ce094aeb986e7d6e4fd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:55.718511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:55.732747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 873/1756] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:55 2022] DELETE /identity/v3/users/9cc7585a11fc4ce094aeb986e7d6e4fd => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:24:55.901775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edef0e-6c3a-48a8-9644-4351b4db7c41 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:55.903603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-37edef0e-6c3a-48a8-9644-4351b4db7c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:55.921797 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edef0e-6c3a-48a8-9644-4351b4db7c41 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:55.987825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:55.988708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:55.988991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:55.989271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] PATH_INFO: `/v3/projects/99df372c1dee4f8b8f70fe5f08494d60` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:55.990492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=99df372c1dee4f8b8f70fe5f08494d60)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:55.992629 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:56.099092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99df372c1dee4f8b8f70fe5f08494d60', 'request_id': 'req-ea82389a-b6a5-45e5-a9aa-a64126fffffe'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.116989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99df372c1dee4f8b8f70fe5f08494d60', 'request_id': 'req-ea82389a-b6a5-45e5-a9aa-a64126fffffe'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.222217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99df372c1dee4f8b8f70fe5f08494d60', 'request_id': 'req-ea82389a-b6a5-45e5-a9aa-a64126fffffe'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.234970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] The token cache is being invalidate because project 99df372c1dee4f8b8f70fe5f08494d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:24:56.235212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.235884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.237456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 884/1757] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:24:55 2022] DELETE /identity/v3/projects/99df372c1dee4f8b8f70fe5f08494d60 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:56.241570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:56.243033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:56.256141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5532-959c-4c1f-b0ba-c17e96c92492 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:56.323455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:56.324849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:56.325303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:56.325747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] PATH_INFO: `/v3/users/60e5b0fc11bf4982aae936e36a36ddb8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:56.327620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e5b0fc11bf4982aae936e36a36ddb8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:56.330952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:56.391850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:56.394487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:56.431142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82389a-b6a5-45e5-a9aa-a64126fffffe admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:56.436460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:56.437461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:56.439089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:56.439089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] PATH_INFO: `/v3/projects/b1c7c25d175a4fd8aa1f2034b9e6d6da` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:56.439521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c7c25d175a4fd8aa1f2034b9e6d6da)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:56.441515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:56.683794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c7c25d175a4fd8aa1f2034b9e6d6da', 'request_id': 'req-93e3ae73-bbd6-4787-b211-e46336eaa1f5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.692238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e5b0fc11bf4982aae936e36a36ddb8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.692913 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e5b0fc11bf4982aae936e36a36ddb8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.696938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c7c25d175a4fd8aa1f2034b9e6d6da', 'request_id': 'req-93e3ae73-bbd6-4787-b211-e46336eaa1f5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.707795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e5b0fc11bf4982aae936e36a36ddb8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.708519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e5b0fc11bf4982aae936e36a36ddb8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.816834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c7c25d175a4fd8aa1f2034b9e6d6da', 'request_id': 'req-93e3ae73-bbd6-4787-b211-e46336eaa1f5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.835377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] The token cache is being invalidate because project b1c7c25d175a4fd8aa1f2034b9e6d6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:24:56.835665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.836369 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-93e3ae73-bbd6-4787-b211-e46336eaa1f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.838140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 885/1758] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:24:56 2022] DELETE /identity/v3/projects/b1c7c25d175a4fd8aa1f2034b9e6d6da => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:56.843856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e2c87-09dc-4cc4-ae02-b213ba9a4170 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:56.844218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e2c87-09dc-4cc4-ae02-b213ba9a4170 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:56.844496 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e2c87-09dc-4cc4-ae02-b213ba9a4170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:56.862542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e77e2c87-09dc-4cc4-ae02-b213ba9a4170 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:56.866899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e77e2c87-09dc-4cc4-ae02-b213ba9a4170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:56.924814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 886/1759] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:56.928754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e5b0fc11bf4982aae936e36a36ddb8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:56.932941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30697092-0e0f-4ba0-b6b8-931e638ed2ba None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:56.935825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30697092-0e0f-4ba0-b6b8-931e638ed2ba None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:56.935825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30697092-0e0f-4ba0-b6b8-931e638ed2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:56.949458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 874/1760] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:56 2022] DELETE /identity/v3/users/60e5b0fc11bf4982aae936e36a36ddb8 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:56.956068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-30697092-0e0f-4ba0-b6b8-931e638ed2ba None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:56.957294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-30697092-0e0f-4ba0-b6b8-931e638ed2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:56.991463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:57.032740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 887/1761] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:57.037106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.038705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:57.052179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5329ee-ae15-489b-b488-f20db20431e0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.057077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87acd1-66d3-4a35-b6fa-1c67d11882da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.057835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87acd1-66d3-4a35-b6fa-1c67d11882da admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.058112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87acd1-66d3-4a35-b6fa-1c67d11882da admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.058438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87acd1-66d3-4a35-b6fa-1c67d11882da admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.059189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87acd1-66d3-4a35-b6fa-1c67d11882da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:57.059504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87acd1-66d3-4a35-b6fa-1c67d11882da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.061110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87acd1-66d3-4a35-b6fa-1c67d11882da admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:57.073692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 875/1762] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:24:57.079452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30697092-0e0f-4ba0-b6b8-931e638ed2ba None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.088595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30697092-0e0f-4ba0-b6b8-931e638ed2ba None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.093316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96113ee8-0b33-4c7b-be0c-f135fc6a1f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.093970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96113ee8-0b33-4c7b-be0c-f135fc6a1f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.094251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96113ee8-0b33-4c7b-be0c-f135fc6a1f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.094577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96113ee8-0b33-4c7b-be0c-f135fc6a1f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.095271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96113ee8-0b33-4c7b-be0c-f135fc6a1f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.097168 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96113ee8-0b33-4c7b-be0c-f135fc6a1f6e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:57.196178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 888/1763] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:57 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:57.206661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87acd1-66d3-4a35-b6fa-1c67d11882da admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.228430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87acd1-66d3-4a35-b6fa-1c67d11882da admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.328931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344ad9b-5a84-432c-89b4-d0194e130d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.329753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344ad9b-5a84-432c-89b4-d0194e130d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.330044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344ad9b-5a84-432c-89b4-d0194e130d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.330350 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344ad9b-5a84-432c-89b4-d0194e130d4a admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.331288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344ad9b-5a84-432c-89b4-d0194e130d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.333353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344ad9b-5a84-432c-89b4-d0194e130d4a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:57.431039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 876/1764] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:57 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:57.433933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96113ee8-0b33-4c7b-be0c-f135fc6a1f6e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.434934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344ad9b-5a84-432c-89b4-d0194e130d4a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.437521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-96113ee8-0b33-4c7b-be0c-f135fc6a1f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:57.448204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344ad9b-5a84-432c-89b4-d0194e130d4a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.454924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabe5ad-874a-4b08-9181-a90d64bc6a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.455152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96113ee8-0b33-4c7b-be0c-f135fc6a1f6e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.455660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabe5ad-874a-4b08-9181-a90d64bc6a56 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.455879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabe5ad-874a-4b08-9181-a90d64bc6a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.456156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabe5ad-874a-4b08-9181-a90d64bc6a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.456887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabe5ad-874a-4b08-9181-a90d64bc6a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.458851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabe5ad-874a-4b08-9181-a90d64bc6a56 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:57.460218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.460606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.460891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.461171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] PATH_INFO: `/v3/projects/f6e8ade56f0b449cbdbbdaac0daff03e` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.462386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e8ade56f0b449cbdbbdaac0daff03e)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.464370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:57.476591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 877/1765] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:24:57.477267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabe5ad-874a-4b08-9181-a90d64bc6a56 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.488151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabe5ad-874a-4b08-9181-a90d64bc6a56 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.493012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec767a3f-6783-45ae-9fe9-f3caffd56570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.493677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767a3f-6783-45ae-9fe9-f3caffd56570 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.494053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767a3f-6783-45ae-9fe9-f3caffd56570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.494355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec767a3f-6783-45ae-9fe9-f3caffd56570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.501396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec767a3f-6783-45ae-9fe9-f3caffd56570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.501396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec767a3f-6783-45ae-9fe9-f3caffd56570 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:57.508145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 878/1766] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:24:57.512476 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec767a3f-6783-45ae-9fe9-f3caffd56570 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.522898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec767a3f-6783-45ae-9fe9-f3caffd56570 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.527466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4689ea6-bc09-473a-9f66-39c2f712c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.528335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4689ea6-bc09-473a-9f66-39c2f712c672 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.528613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4689ea6-bc09-473a-9f66-39c2f712c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.530022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4689ea6-bc09-473a-9f66-39c2f712c672 admin admin] PATH_INFO: `/v3/projects/d88521cded214a5b8cca6d49c371cf81/users/b4764d2bc8984cfb8eaa8e861f3a7810/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.544126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4689ea6-bc09-473a-9f66-39c2f712c672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88521cded214a5b8cca6d49c371cf81, user_id=b4764d2bc8984cfb8eaa8e861f3a7810, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.548077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4689ea6-bc09-473a-9f66-39c2f712c672 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:57.702512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 879/1767] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:57 2022] PUT /identity/v3/projects/d88521cded214a5b8cca6d49c371cf81/users/b4764d2bc8984cfb8eaa8e861f3a7810/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:24:57.703894 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e8ade56f0b449cbdbbdaac0daff03e', 'request_id': 'req-aaa4d320-2beb-433b-a458-7cd310136060'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:57.705520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9dce-f32f-4ffe-82ef-2e0c71e6d1d8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.705520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9dce-f32f-4ffe-82ef-2e0c71e6d1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.705520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c9dce-f32f-4ffe-82ef-2e0c71e6d1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.710342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e8ade56f0b449cbdbbdaac0daff03e', 'request_id': 'req-aaa4d320-2beb-433b-a458-7cd310136060'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:57.741040 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2b7c9dce-f32f-4ffe-82ef-2e0c71e6d1d8 None None] MFA Rules not processed for user `b4764d2bc8984cfb8eaa8e861f3a7810`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:57.742286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2b7c9dce-f32f-4ffe-82ef-2e0c71e6d1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:57.816910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 880/1768] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:57.821591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c9dce-f32f-4ffe-82ef-2e0c71e6d1d8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.856697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e8ade56f0b449cbdbbdaac0daff03e', 'request_id': 'req-aaa4d320-2beb-433b-a458-7cd310136060'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:57.883785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] The token cache is being invalidate because project f6e8ade56f0b449cbdbbdaac0daff03e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:24:57.884202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:57.885397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-aaa4d320-2beb-433b-a458-7cd310136060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:24:57.892126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 889/1769] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:24:57 2022] DELETE /identity/v3/projects/f6e8ade56f0b449cbdbbdaac0daff03e => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:57.895648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbb56c-e628-4da0-9827-ccc4db9a59e1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.895941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbb56c-e628-4da0-9827-ccc4db9a59e1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.896395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbb56c-e628-4da0-9827-ccc4db9a59e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.910587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c9dce-f32f-4ffe-82ef-2e0c71e6d1d8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.915405 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc520c-19af-44a9-943d-7e1295817d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.916158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc520c-19af-44a9-943d-7e1295817d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.916361 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc520c-19af-44a9-943d-7e1295817d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.916638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc520c-19af-44a9-943d-7e1295817d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.917085 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6ddbb56c-e628-4da0-9827-ccc4db9a59e1 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:57.917338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc520c-19af-44a9-943d-7e1295817d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.918318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6ddbb56c-e628-4da0-9827-ccc4db9a59e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:57.919359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc520c-19af-44a9-943d-7e1295817d9e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:57.928978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 890/1770] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:57.937950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ebba1-30d0-4e7e-8592-5807debbb5e5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.938429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ebba1-30d0-4e7e-8592-5807debbb5e5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.938554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ebba1-30d0-4e7e-8592-5807debbb5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.961079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5d2ebba1-30d0-4e7e-8592-5807debbb5e5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:57.962273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5d2ebba1-30d0-4e7e-8592-5807debbb5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:57.971358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 891/1771] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:57.975642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ebba1-30d0-4e7e-8592-5807debbb5e5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:57.987976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ebba1-30d0-4e7e-8592-5807debbb5e5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:57.993113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40601ab1-ad07-4988-91dc-bfd38ff73b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:57.993818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40601ab1-ad07-4988-91dc-bfd38ff73b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:57.994104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40601ab1-ad07-4988-91dc-bfd38ff73b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:57.994436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40601ab1-ad07-4988-91dc-bfd38ff73b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:57.995242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40601ab1-ad07-4988-91dc-bfd38ff73b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:24:57.995618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40601ab1-ad07-4988-91dc-bfd38ff73b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:57.997259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40601ab1-ad07-4988-91dc-bfd38ff73b3a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.010513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 892/1772] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:24:58.014974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40601ab1-ad07-4988-91dc-bfd38ff73b3a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.028899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40601ab1-ad07-4988-91dc-bfd38ff73b3a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.029850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 881/1773] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:57 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:24:58.035176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc520c-19af-44a9-943d-7e1295817d9e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.036990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3dcc520c-19af-44a9-943d-7e1295817d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:58.097472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce0c58-7745-4e9f-8638-9bdc5195e032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.098220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce0c58-7745-4e9f-8638-9bdc5195e032 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.098610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce0c58-7745-4e9f-8638-9bdc5195e032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.098964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce0c58-7745-4e9f-8638-9bdc5195e032 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.099760 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce0c58-7745-4e9f-8638-9bdc5195e032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.101771 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce0c58-7745-4e9f-8638-9bdc5195e032 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.112649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc520c-19af-44a9-943d-7e1295817d9e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.118576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933bdd1-8c3e-46d9-ad11-a93aa48c9e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.119451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933bdd1-8c3e-46d9-ad11-a93aa48c9e64 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.120201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933bdd1-8c3e-46d9-ad11-a93aa48c9e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.120487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933bdd1-8c3e-46d9-ad11-a93aa48c9e64 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.128961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933bdd1-8c3e-46d9-ad11-a93aa48c9e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.128961 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933bdd1-8c3e-46d9-ad11-a93aa48c9e64 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.154044 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:58.229573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 893/1774] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:58 2022] POST /identity/v3/projects => generated 407 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:58.233885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce0c58-7745-4e9f-8638-9bdc5195e032 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.251468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 882/1775] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:58 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:58.257704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933bdd1-8c3e-46d9-ad11-a93aa48c9e64 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.268445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933bdd1-8c3e-46d9-ad11-a93aa48c9e64 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.299998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce0c58-7745-4e9f-8638-9bdc5195e032 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.305176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac3041-c377-453f-a5ae-e1ae5e971560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.306031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac3041-c377-453f-a5ae-e1ae5e971560 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.306378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac3041-c377-453f-a5ae-e1ae5e971560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.306843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac3041-c377-453f-a5ae-e1ae5e971560 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.312642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac3041-c377-453f-a5ae-e1ae5e971560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.312642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac3041-c377-453f-a5ae-e1ae5e971560 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.341646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d9c7e-e2d5-4841-b17f-b7476dd5d9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.341646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d9c7e-e2d5-4841-b17f-b7476dd5d9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.341646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d9c7e-e2d5-4841-b17f-b7476dd5d9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.341646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d9c7e-e2d5-4841-b17f-b7476dd5d9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.342398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d9c7e-e2d5-4841-b17f-b7476dd5d9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.343585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d9c7e-e2d5-4841-b17f-b7476dd5d9c6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.357492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 883/1776] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:24:58.359123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d9c7e-e2d5-4841-b17f-b7476dd5d9c6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.370197 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d9c7e-e2d5-4841-b17f-b7476dd5d9c6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.377413 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376a2c2-4a86-43f6-908a-750c2907ff78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.377413 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376a2c2-4a86-43f6-908a-750c2907ff78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.377413 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376a2c2-4a86-43f6-908a-750c2907ff78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.379477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376a2c2-4a86-43f6-908a-750c2907ff78 admin admin] PATH_INFO: `/v3/projects/e3e60e60ff684d84858a2bd9d159d500/users/2231c82f927d4aeb878a071e9133f6f4/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.398204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376a2c2-4a86-43f6-908a-750c2907ff78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e60e60ff684d84858a2bd9d159d500, user_id=2231c82f927d4aeb878a071e9133f6f4, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.401260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376a2c2-4a86-43f6-908a-750c2907ff78 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.447625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 894/1777] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:58 2022] POST /identity/v3/users => generated 304 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:24:58.455931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac3041-c377-453f-a5ae-e1ae5e971560 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.467388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac3041-c377-453f-a5ae-e1ae5e971560 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.473903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85753aa-c19c-468d-ac61-698a357e1929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.474631 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85753aa-c19c-468d-ac61-698a357e1929 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.474936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85753aa-c19c-468d-ac61-698a357e1929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.475327 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85753aa-c19c-468d-ac61-698a357e1929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.476121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85753aa-c19c-468d-ac61-698a357e1929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.477775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85753aa-c19c-468d-ac61-698a357e1929 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.493497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 895/1778] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:24:58.499446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85753aa-c19c-468d-ac61-698a357e1929 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.510267 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85753aa-c19c-468d-ac61-698a357e1929 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.516910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2fead-2f44-446c-a507-39574715e440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.517871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2fead-2f44-446c-a507-39574715e440 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.518489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2fead-2f44-446c-a507-39574715e440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.518917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2fead-2f44-446c-a507-39574715e440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.519834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2fead-2f44-446c-a507-39574715e440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.521597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2fead-2f44-446c-a507-39574715e440 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.538271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 896/1779] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:24:58.553082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2fead-2f44-446c-a507-39574715e440 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.553790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 884/1780] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:58 2022] PUT /identity/v3/projects/e3e60e60ff684d84858a2bd9d159d500/users/2231c82f927d4aeb878a071e9133f6f4/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:58.555193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376a2c2-4a86-43f6-908a-750c2907ff78 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.561739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2fead-2f44-446c-a507-39574715e440 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.630533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ef6fa-7488-4140-ab01-47dd4edc7d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.631795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ef6fa-7488-4140-ab01-47dd4edc7d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.632484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ef6fa-7488-4140-ab01-47dd4edc7d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.632923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ef6fa-7488-4140-ab01-47dd4edc7d52 admin admin] PATH_INFO: `/v3/projects/7465e40745404914af7df65f5c3b0bcb/users/ea9c23b941ed475db68f6256fef284a0/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.644735 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376a2c2-4a86-43f6-908a-750c2907ff78 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.650305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436a796-abc8-4c22-b240-9a64d26ae93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.651108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436a796-abc8-4c22-b240-9a64d26ae93e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.651478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436a796-abc8-4c22-b240-9a64d26ae93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.651863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436a796-abc8-4c22-b240-9a64d26ae93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.652729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436a796-abc8-4c22-b240-9a64d26ae93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.654895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436a796-abc8-4c22-b240-9a64d26ae93e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.662526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ef6fa-7488-4140-ab01-47dd4edc7d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7465e40745404914af7df65f5c3b0bcb, user_id=ea9c23b941ed475db68f6256fef284a0, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.665272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ef6fa-7488-4140-ab01-47dd4edc7d52 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.666841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 885/1781] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:58.670885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436a796-abc8-4c22-b240-9a64d26ae93e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.679772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:58.683721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436a796-abc8-4c22-b240-9a64d26ae93e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.688833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3d004-718c-49f5-9b0c-fa6b953c7156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.689772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3d004-718c-49f5-9b0c-fa6b953c7156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.690060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3d004-718c-49f5-9b0c-fa6b953c7156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.690339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3d004-718c-49f5-9b0c-fa6b953c7156 admin admin] PATH_INFO: `/v3/projects/e3e60e60ff684d84858a2bd9d159d500/users/2231c82f927d4aeb878a071e9133f6f4/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.692337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3d004-718c-49f5-9b0c-fa6b953c7156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e60e60ff684d84858a2bd9d159d500, user_id=2231c82f927d4aeb878a071e9133f6f4, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:58.695210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3d004-718c-49f5-9b0c-fa6b953c7156 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:58.706218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 886/1782] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:58 2022] PUT /identity/v3/projects/e3e60e60ff684d84858a2bd9d159d500/users/2231c82f927d4aeb878a071e9133f6f4/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:24:58.716532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83d33a-7baa-4f68-976d-d2585738823b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.716828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83d33a-7baa-4f68-976d-d2585738823b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.717114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83d33a-7baa-4f68-976d-d2585738823b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.744675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3f83d33a-7baa-4f68-976d-d2585738823b None None] MFA Rules not processed for user `2231c82f927d4aeb878a071e9133f6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:58.745816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3f83d33a-7baa-4f68-976d-d2585738823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:58.759726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 897/1783] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:58 2022] PUT /identity/v3/projects/7465e40745404914af7df65f5c3b0bcb/users/ea9c23b941ed475db68f6256fef284a0/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:58.776172 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fe247-2a14-4323-a941-93a021e47e7f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.776487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fe247-2a14-4323-a941-93a021e47e7f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.776783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fe247-2a14-4323-a941-93a021e47e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:58.808929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e16fe247-2a14-4323-a941-93a021e47e7f None None] MFA Rules not processed for user `ea9c23b941ed475db68f6256fef284a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:58.810166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e16fe247-2a14-4323-a941-93a021e47e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:58.824439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 887/1784] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:58.836954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83d33a-7baa-4f68-976d-d2585738823b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.838851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3f83d33a-7baa-4f68-976d-d2585738823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:58.900819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 898/1785] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:24:58.902352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fe247-2a14-4323-a941-93a021e47e7f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:58.904175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83d33a-7baa-4f68-976d-d2585738823b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:58.974898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109ae23-0da3-4155-b4d9-0e2147bb1e33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:58.975632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109ae23-0da3-4155-b4d9-0e2147bb1e33 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:58.975914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109ae23-0da3-4155-b4d9-0e2147bb1e33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:58.976191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109ae23-0da3-4155-b4d9-0e2147bb1e33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.006571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fe247-2a14-4323-a941-93a021e47e7f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:59.015673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd2f58-41e7-4c6b-82c1-b47a49ae74c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:59.016846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd2f58-41e7-4c6b-82c1-b47a49ae74c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.017285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd2f58-41e7-4c6b-82c1-b47a49ae74c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.019171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd2f58-41e7-4c6b-82c1-b47a49ae74c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.024215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd2f58-41e7-4c6b-82c1-b47a49ae74c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.024215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd2f58-41e7-4c6b-82c1-b47a49ae74c4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:59.059927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109ae23-0da3-4155-b4d9-0e2147bb1e33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.061431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109ae23-0da3-4155-b4d9-0e2147bb1e33 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:59.075701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 888/1786] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:24:59.131640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 899/1787] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:24:58 2022] POST /identity/v3/projects => generated 409 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:24:59.135743 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109ae23-0da3-4155-b4d9-0e2147bb1e33 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:59.150359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109ae23-0da3-4155-b4d9-0e2147bb1e33 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:59.213697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccb1f1-87ff-4b27-a995-0f998d6fded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:59.214814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccb1f1-87ff-4b27-a995-0f998d6fded7 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.215099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccb1f1-87ff-4b27-a995-0f998d6fded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.215382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccb1f1-87ff-4b27-a995-0f998d6fded7 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.216103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccb1f1-87ff-4b27-a995-0f998d6fded7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.218185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccb1f1-87ff-4b27-a995-0f998d6fded7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:59.254604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd2f58-41e7-4c6b-82c1-b47a49ae74c4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:59.308874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 889/1788] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:24:59 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:24:59.313193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd2f58-41e7-4c6b-82c1-b47a49ae74c4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:59.313563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccb1f1-87ff-4b27-a995-0f998d6fded7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:59.317201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea6390-8124-4cf1-8cf0-95dbd374608c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:59.317844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea6390-8124-4cf1-8cf0-95dbd374608c service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.318174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea6390-8124-4cf1-8cf0-95dbd374608c service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.318678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea6390-8124-4cf1-8cf0-95dbd374608c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.322172 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-06ea6390-8124-4cf1-8cf0-95dbd374608c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:59.323843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccb1f1-87ff-4b27-a995-0f998d6fded7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:59.329045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5947a4-14bf-47e7-97e1-85a3f67b950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:59.329704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5947a4-14bf-47e7-97e1-85a3f67b950a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.329991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5947a4-14bf-47e7-97e1-85a3f67b950a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.330265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5947a4-14bf-47e7-97e1-85a3f67b950a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.331028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5947a4-14bf-47e7-97e1-85a3f67b950a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.332608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5947a4-14bf-47e7-97e1-85a3f67b950a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:59.342781 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 890/1789] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:59.346856 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5947a4-14bf-47e7-97e1-85a3f67b950a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:59.356401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5947a4-14bf-47e7-97e1-85a3f67b950a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:59.361316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf035d76-9a38-4c99-99ae-fbf9d53d157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:59.362678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf035d76-9a38-4c99-99ae-fbf9d53d157a admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.362995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf035d76-9a38-4c99-99ae-fbf9d53d157a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.363276 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf035d76-9a38-4c99-99ae-fbf9d53d157a admin admin] PATH_INFO: `/v3/projects/7f7d0ed2adb5445ea89986dc205d058b/users/0b06c181232a445593ddc2dda9ea9abb/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.380447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf035d76-9a38-4c99-99ae-fbf9d53d157a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7d0ed2adb5445ea89986dc205d058b, user_id=0b06c181232a445593ddc2dda9ea9abb, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.382168 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea6390-8124-4cf1-8cf0-95dbd374608c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.383162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf035d76-9a38-4c99-99ae-fbf9d53d157a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:59.384145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea6390-8124-4cf1-8cf0-95dbd374608c service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:59.393899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 900/1790] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:24:59.488277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 891/1791] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:59 2022] PUT /identity/v3/projects/7f7d0ed2adb5445ea89986dc205d058b/users/0b06c181232a445593ddc2dda9ea9abb/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:24:59.492232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea6390-8124-4cf1-8cf0-95dbd374608c service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:59.504096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea6390-8124-4cf1-8cf0-95dbd374608c service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:59.512067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:59.579547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc22b92-0c52-4ed5-8cb7-ba02755ce26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:59.579547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc22b92-0c52-4ed5-8cb7-ba02755ce26d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.579547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc22b92-0c52-4ed5-8cb7-ba02755ce26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.579547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc22b92-0c52-4ed5-8cb7-ba02755ce26d admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.579990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc22b92-0c52-4ed5-8cb7-ba02755ce26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.581584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc22b92-0c52-4ed5-8cb7-ba02755ce26d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:59.595449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 901/1792] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:24:59.599660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf035d76-9a38-4c99-99ae-fbf9d53d157a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:59.610640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf035d76-9a38-4c99-99ae-fbf9d53d157a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:59.615599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8ff2d-625c-42a5-8420-9731318f07c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:59.616503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8ff2d-625c-42a5-8420-9731318f07c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.616773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8ff2d-625c-42a5-8420-9731318f07c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.617050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8ff2d-625c-42a5-8420-9731318f07c9 admin admin] PATH_INFO: `/v3/projects/7f7d0ed2adb5445ea89986dc205d058b/users/0b06c181232a445593ddc2dda9ea9abb/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.619114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8ff2d-625c-42a5-8420-9731318f07c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7d0ed2adb5445ea89986dc205d058b, user_id=0b06c181232a445593ddc2dda9ea9abb, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.622495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8ff2d-625c-42a5-8420-9731318f07c9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:59.635182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 892/1793] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:24:59 2022] PUT /identity/v3/projects/7f7d0ed2adb5445ea89986dc205d058b/users/0b06c181232a445593ddc2dda9ea9abb/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:24:59.646850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de68f-d08a-4709-9977-45c7f44cd088 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.647613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de68f-d08a-4709-9977-45c7f44cd088 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.647995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de68f-d08a-4709-9977-45c7f44cd088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.657153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:24:59.681052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-039de68f-d08a-4709-9977-45c7f44cd088 None None] MFA Rules not processed for user `0b06c181232a445593ddc2dda9ea9abb`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:24:59.682264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-039de68f-d08a-4709-9977-45c7f44cd088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:59.760312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 902/1794] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:24:59 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:24:59.771288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8ff2d-625c-42a5-8420-9731318f07c9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:59.773104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-91f8ff2d-625c-42a5-8420-9731318f07c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:24:59.831881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8ff2d-625c-42a5-8420-9731318f07c9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:59.835460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7a22c-42f7-45c6-88e0-dd8a63f67a36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:59.836097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7a22c-42f7-45c6-88e0-dd8a63f67a36 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.836378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7a22c-42f7-45c6-88e0-dd8a63f67a36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.839916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7a22c-42f7-45c6-88e0-dd8a63f67a36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.845838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7a22c-42f7-45c6-88e0-dd8a63f67a36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.847289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7a22c-42f7-45c6-88e0-dd8a63f67a36 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:24:59.856713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 893/1795] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:24:59 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:24:59.910354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039de68f-d08a-4709-9977-45c7f44cd088 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:24:59.985264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039de68f-d08a-4709-9977-45c7f44cd088 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:24:59.989882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:24:59.990794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:24:59.991078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:24:59.991359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] PATH_INFO: `/v3/users/b4764d2bc8984cfb8eaa8e861f3a7810` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:24:59.992555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4764d2bc8984cfb8eaa8e861f3a7810)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:24:59.994525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:00.211860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4764d2bc8984cfb8eaa8e861f3a7810'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:00.220237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4764d2bc8984cfb8eaa8e861f3a7810'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:00.220942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4764d2bc8984cfb8eaa8e861f3a7810'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:00.221555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4764d2bc8984cfb8eaa8e861f3a7810'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:00.282369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7a22c-42f7-45c6-88e0-dd8a63f67a36 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:00.291013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7a22c-42f7-45c6-88e0-dd8a63f67a36 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:00.295065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:00.307670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:00.307670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:00.307670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:00.307670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:00.338015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-04a0c8da-4b82-446b-80e7-8aa0be1b40b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4764d2bc8984cfb8eaa8e861f3a7810'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:00.352815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 903/1796] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:24:59 2022] DELETE /identity/v3/users/b4764d2bc8984cfb8eaa8e861f3a7810 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:00.359156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cbd74-cdbb-4d9a-acf6-814a4445eccf None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:00.360081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cbd74-cdbb-4d9a-acf6-814a4445eccf None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:00.360531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cbd74-cdbb-4d9a-acf6-814a4445eccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:00.381435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b38cbd74-cdbb-4d9a-acf6-814a4445eccf None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:00.382712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b38cbd74-cdbb-4d9a-acf6-814a4445eccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:00.400471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:00.401988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:00.413793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 894/1797] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:25:00.462217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 904/1798] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:00.474193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:00.475783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:00.554127 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07789aa3-fe1b-4513-ae74-54dccd3aa8cc service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:00.619454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee2ad-a52d-4fbc-8558-55b6bfc152a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:00.620900 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee2ad-a52d-4fbc-8558-55b6bfc152a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:00.620900 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee2ad-a52d-4fbc-8558-55b6bfc152a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:00.620900 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee2ad-a52d-4fbc-8558-55b6bfc152a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:00.634631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ee2ad-a52d-4fbc-8558-55b6bfc152a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:00.637343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557ee2ad-a52d-4fbc-8558-55b6bfc152a9 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:00.649322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 895/1799] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:01.028741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:01.152896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38cbd74-cdbb-4d9a-acf6-814a4445eccf None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:01.154433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b38cbd74-cdbb-4d9a-acf6-814a4445eccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:01.169058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38cbd74-cdbb-4d9a-acf6-814a4445eccf None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:01.184637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:01.185592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:01.185903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:01.186189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] PATH_INFO: `/v3/projects/d88521cded214a5b8cca6d49c371cf81` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:01.188353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d88521cded214a5b8cca6d49c371cf81)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:01.189748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:01.363465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88521cded214a5b8cca6d49c371cf81', 'request_id': 'req-ec93ae48-5f0e-4fc7-9355-0b413cf67051'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:01.369781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88521cded214a5b8cca6d49c371cf81', 'request_id': 'req-ec93ae48-5f0e-4fc7-9355-0b413cf67051'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:01.470993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88521cded214a5b8cca6d49c371cf81', 'request_id': 'req-ec93ae48-5f0e-4fc7-9355-0b413cf67051'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:01.485919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] The token cache is being invalidate because project d88521cded214a5b8cca6d49c371cf81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:01.486294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:01.486974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:01.488564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 905/1800] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:01 2022] DELETE /identity/v3/projects/d88521cded214a5b8cca6d49c371cf81 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:01.493243 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee2ad-a52d-4fbc-8558-55b6bfc152a9 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:01.504318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee2ad-a52d-4fbc-8558-55b6bfc152a9 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:01.585646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:01.585646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:01.585646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:01.585646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] PATH_INFO: `/v3/users/2231c82f927d4aeb878a071e9133f6f4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:01.586824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2231c82f927d4aeb878a071e9133f6f4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:01.588343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:01.871945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2231c82f927d4aeb878a071e9133f6f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:01.872631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2231c82f927d4aeb878a071e9133f6f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:01.889443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2231c82f927d4aeb878a071e9133f6f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:01.890956 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2231c82f927d4aeb878a071e9133f6f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:02.060138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fc5da8db-603d-48c4-bb18-06d74a4a8788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2231c82f927d4aeb878a071e9133f6f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:02.076073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 896/1801] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:01 2022] DELETE /identity/v3/users/2231c82f927d4aeb878a071e9133f6f4 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:02.417252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:02.691563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:02.718579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:02.718579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93ae48-5f0e-4fc7-9355-0b413cf67051 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:02.773890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:02.774975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:02.775260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:02.775538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] PATH_INFO: `/v3/projects/e3e60e60ff684d84858a2bd9d159d500` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:02.776821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e60e60ff684d84858a2bd9d159d500)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:02.778861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:03.004856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e60e60ff684d84858a2bd9d159d500', 'request_id': 'req-19559a30-fb88-4611-beda-b459d98bf445'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:03.006125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e60e60ff684d84858a2bd9d159d500', 'request_id': 'req-19559a30-fb88-4611-beda-b459d98bf445'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:03.425879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e60e60ff684d84858a2bd9d159d500', 'request_id': 'req-19559a30-fb88-4611-beda-b459d98bf445'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:03.438460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] The token cache is being invalidate because project e3e60e60ff684d84858a2bd9d159d500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:03.438727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:03.439401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-19559a30-fb88-4611-beda-b459d98bf445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:03.441142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 906/1802] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:02 2022] DELETE /identity/v3/projects/e3e60e60ff684d84858a2bd9d159d500 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:03.446226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b5673-b688-42fe-8a87-ffd4f2be56b8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.446775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b5673-b688-42fe-8a87-ffd4f2be56b8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.446871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b5673-b688-42fe-8a87-ffd4f2be56b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.465354 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-764b5673-b688-42fe-8a87-ffd4f2be56b8 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:03.466610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-764b5673-b688-42fe-8a87-ffd4f2be56b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:03.525951 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 897/1803] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:03.533974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a014db-556d-4355-b8e0-b40baad8f746 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.534264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a014db-556d-4355-b8e0-b40baad8f746 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.534576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a014db-556d-4355-b8e0-b40baad8f746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.555765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d4a014db-556d-4355-b8e0-b40baad8f746 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:03.557157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d4a014db-556d-4355-b8e0-b40baad8f746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:03.567022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 907/1804] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:03.571303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b5673-b688-42fe-8a87-ffd4f2be56b8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:03.585299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b5673-b688-42fe-8a87-ffd4f2be56b8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:03.586231 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cd96d-3d4a-4589-a122-e8f2bc62be36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:03.586986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cd96d-3d4a-4589-a122-e8f2bc62be36 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.587272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cd96d-3d4a-4589-a122-e8f2bc62be36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.587545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cd96d-3d4a-4589-a122-e8f2bc62be36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.588540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cd96d-3d4a-4589-a122-e8f2bc62be36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:03.588656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cd96d-3d4a-4589-a122-e8f2bc62be36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:03.590390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4cd96d-3d4a-4589-a122-e8f2bc62be36 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:03.603020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 898/1805] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:03.607283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a014db-556d-4355-b8e0-b40baad8f746 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:03.617702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a014db-556d-4355-b8e0-b40baad8f746 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:03.622750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547db8a-f65f-4a85-972e-bbb0c8fb1ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:03.623437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547db8a-f65f-4a85-972e-bbb0c8fb1ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.623725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547db8a-f65f-4a85-972e-bbb0c8fb1ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.624667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547db8a-f65f-4a85-972e-bbb0c8fb1ad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.625613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547db8a-f65f-4a85-972e-bbb0c8fb1ad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:03.627590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547db8a-f65f-4a85-972e-bbb0c8fb1ad8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:03.662948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:03.715827 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 908/1806] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:03 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:03.720559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cd96d-3d4a-4589-a122-e8f2bc62be36 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:03.731434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cd96d-3d4a-4589-a122-e8f2bc62be36 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:03.752129 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e0615-9c7b-43c8-853a-caa3c62ec199 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.752129 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e0615-9c7b-43c8-853a-caa3c62ec199 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.752650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e0615-9c7b-43c8-853a-caa3c62ec199 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.757273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 909/1807] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:25:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:25:03.771624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eaca2-2875-47a9-bdc8-02fba0379e8c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.771948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eaca2-2875-47a9-bdc8-02fba0379e8c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.772286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eaca2-2875-47a9-bdc8-02fba0379e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.792591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ff6eaca2-2875-47a9-bdc8-02fba0379e8c None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:03.793873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ff6eaca2-2875-47a9-bdc8-02fba0379e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:03.796551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7641fa6-f328-4f14-a23a-5de78876c8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:03.797520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7641fa6-f328-4f14-a23a-5de78876c8ed admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.797819 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7641fa6-f328-4f14-a23a-5de78876c8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.798093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7641fa6-f328-4f14-a23a-5de78876c8ed admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.799098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7641fa6-f328-4f14-a23a-5de78876c8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:03.801021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7641fa6-f328-4f14-a23a-5de78876c8ed admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:03.876221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 899/1808] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:03 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:03.877805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7641fa6-f328-4f14-a23a-5de78876c8ed admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:03.879061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 910/1809] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:03.900688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7641fa6-f328-4f14-a23a-5de78876c8ed admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:03.908768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d00318-404a-409b-9422-b248409552ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:03.910431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d00318-404a-409b-9422-b248409552ae admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.910905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d00318-404a-409b-9422-b248409552ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.911182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d00318-404a-409b-9422-b248409552ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.911926 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d00318-404a-409b-9422-b248409552ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:03.914026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d00318-404a-409b-9422-b248409552ae admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:03.930501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 900/1810] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:03.934596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eaca2-2875-47a9-bdc8-02fba0379e8c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:03.948682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6eaca2-2875-47a9-bdc8-02fba0379e8c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:03.950113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff64e29-4952-42a2-9bcd-2b8637550e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:03.952335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff64e29-4952-42a2-9bcd-2b8637550e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.952617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff64e29-4952-42a2-9bcd-2b8637550e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.952907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff64e29-4952-42a2-9bcd-2b8637550e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:03.953648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff64e29-4952-42a2-9bcd-2b8637550e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:03.955315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff64e29-4952-42a2-9bcd-2b8637550e3b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:03.966991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 911/1811] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:03.971503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d00318-404a-409b-9422-b248409552ae admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:03.982046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d00318-404a-409b-9422-b248409552ae admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:03.987495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0229e23-8e90-4ffb-97d6-0855a3a591fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:03.988384 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0229e23-8e90-4ffb-97d6-0855a3a591fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:03.988672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0229e23-8e90-4ffb-97d6-0855a3a591fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:03.988949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0229e23-8e90-4ffb-97d6-0855a3a591fc admin admin] PATH_INFO: `/v3/projects/2eedaeb1403d496da5abbba1808deea2/users/41ae3e565d1f4ef28c1f285f610656f0/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:04.005053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0229e23-8e90-4ffb-97d6-0855a3a591fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eedaeb1403d496da5abbba1808deea2, user_id=41ae3e565d1f4ef28c1f285f610656f0, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:04.007787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0229e23-8e90-4ffb-97d6-0855a3a591fc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:04.065991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff64e29-4952-42a2-9bcd-2b8637550e3b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:04.068517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-eff64e29-4952-42a2-9bcd-2b8637550e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:04.089681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff64e29-4952-42a2-9bcd-2b8637550e3b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:04.097449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2554a7-ceab-4620-abcb-0e773f316d59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:04.098122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2554a7-ceab-4620-abcb-0e773f316d59 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:04.098442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2554a7-ceab-4620-abcb-0e773f316d59 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:04.098728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2554a7-ceab-4620-abcb-0e773f316d59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:04.109085 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2554a7-ceab-4620-abcb-0e773f316d59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:04.110980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2554a7-ceab-4620-abcb-0e773f316d59 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:04.122177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 912/1812] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:25:04.150690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 901/1813] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:03 2022] PUT /identity/v3/projects/2eedaeb1403d496da5abbba1808deea2/users/41ae3e565d1f4ef28c1f285f610656f0/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:04.154095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f1d1d-446b-469c-888b-fe05718ae335 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:04.154612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f1d1d-446b-469c-888b-fe05718ae335 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:04.156079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f1d1d-446b-469c-888b-fe05718ae335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:04.199977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ed5f1d1d-446b-469c-888b-fe05718ae335 None None] MFA Rules not processed for user `41ae3e565d1f4ef28c1f285f610656f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:04.201275 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ed5f1d1d-446b-469c-888b-fe05718ae335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:04.293386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 913/1814] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:04.304438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0229e23-8e90-4ffb-97d6-0855a3a591fc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:04.306260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f0229e23-8e90-4ffb-97d6-0855a3a591fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:04.365629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0229e23-8e90-4ffb-97d6-0855a3a591fc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:04.369336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4befe-c6b2-4b15-b742-0cdbf62e1941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:04.373419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4befe-c6b2-4b15-b742-0cdbf62e1941 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:04.373419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4befe-c6b2-4b15-b742-0cdbf62e1941 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:04.373419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4befe-c6b2-4b15-b742-0cdbf62e1941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:04.379759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4befe-c6b2-4b15-b742-0cdbf62e1941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:04.381192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4befe-c6b2-4b15-b742-0cdbf62e1941 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:04.391269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 902/1815] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:06.253293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:07.384294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:08.982876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:09.239934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:09.488281 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:09.757277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:09.791375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:10.048456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:10.124719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:10.637652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:11.385395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:12.254262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:12.828566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:13.257205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:18.371634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f1d1d-446b-469c-888b-fe05718ae335 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:18.373618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ed5f1d1d-446b-469c-888b-fe05718ae335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:18.391530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:18.451874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f1d1d-446b-469c-888b-fe05718ae335 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:18.458118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:18.459222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:18.459516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:18.459796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] PATH_INFO: `/v3/users/ea9c23b941ed475db68f6256fef284a0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:18.461047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9c23b941ed475db68f6256fef284a0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:18.463217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:18.700662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9c23b941ed475db68f6256fef284a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:18.707834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9c23b941ed475db68f6256fef284a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:18.708525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9c23b941ed475db68f6256fef284a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:18.709140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9c23b941ed475db68f6256fef284a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:18.805110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9c23b941ed475db68f6256fef284a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:18.823967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 914/1816] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:18 2022] DELETE /identity/v3/users/ea9c23b941ed475db68f6256fef284a0 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:18.829030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7beddd-532d-42c6-ac16-3606e32850c9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:18.829508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7beddd-532d-42c6-ac16-3606e32850c9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:18.829602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7beddd-532d-42c6-ac16-3606e32850c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:18.841868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:18.847975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-da7beddd-532d-42c6-ac16-3606e32850c9 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:18.849522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-da7beddd-532d-42c6-ac16-3606e32850c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:18.913308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 903/1817] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:18.923827 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:18.925446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:18.990668 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae431fd-e3a1-4995-8c68-36233693455b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:18.994186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed985fb9-3896-4b64-b791-b7d42690d787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:18.995008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed985fb9-3896-4b64-b791-b7d42690d787 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:18.995292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed985fb9-3896-4b64-b791-b7d42690d787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:18.995568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed985fb9-3896-4b64-b791-b7d42690d787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:19.007021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed985fb9-3896-4b64-b791-b7d42690d787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:19.008879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed985fb9-3896-4b64-b791-b7d42690d787 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:19.032176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 915/1818] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:19.683777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7beddd-532d-42c6-ac16-3606e32850c9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:19.686333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-da7beddd-532d-42c6-ac16-3606e32850c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:19.702778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7beddd-532d-42c6-ac16-3606e32850c9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:19.737010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:19.737010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:19.737010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:19.737010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] PATH_INFO: `/v3/projects/7465e40745404914af7df65f5c3b0bcb` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:19.740232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7465e40745404914af7df65f5c3b0bcb)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:19.744760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:19.902210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7465e40745404914af7df65f5c3b0bcb', 'request_id': 'req-aa74a586-87c5-4e6c-8082-be71e771254d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:19.919252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7465e40745404914af7df65f5c3b0bcb', 'request_id': 'req-aa74a586-87c5-4e6c-8082-be71e771254d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:20.076784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7465e40745404914af7df65f5c3b0bcb', 'request_id': 'req-aa74a586-87c5-4e6c-8082-be71e771254d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:20.092972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] The token cache is being invalidate because project 7465e40745404914af7df65f5c3b0bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:20.093268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:20.093983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:20.095809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 904/1819] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:19 2022] DELETE /identity/v3/projects/7465e40745404914af7df65f5c3b0bcb => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:20.100056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed985fb9-3896-4b64-b791-b7d42690d787 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:20.112526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed985fb9-3896-4b64-b791-b7d42690d787 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:20.149941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:20.185767 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:20.186740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:20.187028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:20.187341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] PATH_INFO: `/v3/users/0b06c181232a445593ddc2dda9ea9abb` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:20.188688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b06c181232a445593ddc2dda9ea9abb)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:20.190955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:20.472345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b06c181232a445593ddc2dda9ea9abb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:20.480813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b06c181232a445593ddc2dda9ea9abb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:20.481645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b06c181232a445593ddc2dda9ea9abb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:20.483119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b06c181232a445593ddc2dda9ea9abb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:20.645438 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b06c181232a445593ddc2dda9ea9abb'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:20.690663 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 916/1820] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:20 2022] DELETE /identity/v3/users/0b06c181232a445593ddc2dda9ea9abb => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:21.080212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:21.083927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:21.111780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74a586-87c5-4e6c-8082-be71e771254d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:21.244239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:21.251149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:21.251149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:21.251149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] PATH_INFO: `/v3/users/41ae3e565d1f4ef28c1f285f610656f0` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:21.251149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ae3e565d1f4ef28c1f285f610656f0)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:21.252458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:21.330602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:21.334377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:21.358257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a5b72-96e3-493f-9cb1-d0860d653c25 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:21.365942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:21.367312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:21.367817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:21.368603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] PATH_INFO: `/v3/projects/7f7d0ed2adb5445ea89986dc205d058b` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:21.371022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f7d0ed2adb5445ea89986dc205d058b)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:21.376422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:21.551828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:21.568583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ae3e565d1f4ef28c1f285f610656f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.569471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ae3e565d1f4ef28c1f285f610656f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.583409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ae3e565d1f4ef28c1f285f610656f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.584308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ae3e565d1f4ef28c1f285f610656f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.589106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f7d0ed2adb5445ea89986dc205d058b', 'request_id': 'req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.595865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f7d0ed2adb5445ea89986dc205d058b', 'request_id': 'req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.703357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-958b70b5-529f-407b-9e50-88829ce4139f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ae3e565d1f4ef28c1f285f610656f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.735383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 905/1821] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:21 2022] DELETE /identity/v3/users/41ae3e565d1f4ef28c1f285f610656f0 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:21.739924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f55c9-ba1a-4d6c-a3fe-ad4db86fc500 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:21.740715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f55c9-ba1a-4d6c-a3fe-ad4db86fc500 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:21.746248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f55c9-ba1a-4d6c-a3fe-ad4db86fc500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:21.787222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a12f55c9-ba1a-4d6c-a3fe-ad4db86fc500 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:21.792882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a12f55c9-ba1a-4d6c-a3fe-ad4db86fc500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:21.806064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f7d0ed2adb5445ea89986dc205d058b', 'request_id': 'req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.829489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] The token cache is being invalidate because project 7f7d0ed2adb5445ea89986dc205d058b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:21.829489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.830555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6a2f3685-d25c-48d5-8c36-5bd8e03e45b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:21.837075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 917/1822] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:21 2022] DELETE /identity/v3/projects/7f7d0ed2adb5445ea89986dc205d058b => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:21.842914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0ee51-e005-4f9c-a3e7-cd9db16797f5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:21.844292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0ee51-e005-4f9c-a3e7-cd9db16797f5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:21.845215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0ee51-e005-4f9c-a3e7-cd9db16797f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:21.884222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c3e0ee51-e005-4f9c-a3e7-cd9db16797f5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:21.886443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c3e0ee51-e005-4f9c-a3e7-cd9db16797f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:21.930534 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 906/1823] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:21.950836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f55c9-ba1a-4d6c-a3fe-ad4db86fc500 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:21.959706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a12f55c9-ba1a-4d6c-a3fe-ad4db86fc500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:21.977456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:22.047523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 918/1824] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:22.055608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e237f-afa7-4a5a-8aa7-62232783cdcd None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:22.056041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e237f-afa7-4a5a-8aa7-62232783cdcd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:22.056433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e237f-afa7-4a5a-8aa7-62232783cdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:22.106500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5e9e237f-afa7-4a5a-8aa7-62232783cdcd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:22.110655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5e9e237f-afa7-4a5a-8aa7-62232783cdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:22.131414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f55c9-ba1a-4d6c-a3fe-ad4db86fc500 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:22.133722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 919/1825] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:22.138351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e237f-afa7-4a5a-8aa7-62232783cdcd None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:22.167148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64266650-2bcf-435d-ad8b-82ba5c9564ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:22.171436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64266650-2bcf-435d-ad8b-82ba5c9564ee service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:22.172317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64266650-2bcf-435d-ad8b-82ba5c9564ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:22.172815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64266650-2bcf-435d-ad8b-82ba5c9564ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:22.192122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e237f-afa7-4a5a-8aa7-62232783cdcd None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:22.193400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64266650-2bcf-435d-ad8b-82ba5c9564ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:22.194964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64266650-2bcf-435d-ad8b-82ba5c9564ee service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:22.210021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7905-f4e3-498b-b095-e6fe0a4b746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:22.210953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 907/1826] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:22.215250 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7905-f4e3-498b-b095-e6fe0a4b746f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:22.215547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7905-f4e3-498b-b095-e6fe0a4b746f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:22.215898 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7905-f4e3-498b-b095-e6fe0a4b746f admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:22.216694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da7905-f4e3-498b-b095-e6fe0a4b746f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:22.217008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da7905-f4e3-498b-b095-e6fe0a4b746f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:22.220335 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da7905-f4e3-498b-b095-e6fe0a4b746f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:22.257000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 920/1827] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:22.261863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64266650-2bcf-435d-ad8b-82ba5c9564ee service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:22.297741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64266650-2bcf-435d-ad8b-82ba5c9564ee service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:22.319431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ba7d4-2976-48a2-8052-545c5837e9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:22.320296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ba7d4-2976-48a2-8052-545c5837e9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:22.320482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ba7d4-2976-48a2-8052-545c5837e9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:22.320744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ba7d4-2976-48a2-8052-545c5837e9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:22.321494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ba7d4-2976-48a2-8052-545c5837e9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:22.324335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ba7d4-2976-48a2-8052-545c5837e9c0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:22.638713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 908/1828] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:22 2022] POST /identity/v3/projects => generated 375 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:22.639882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7905-f4e3-498b-b095-e6fe0a4b746f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:22.717155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7905-f4e3-498b-b095-e6fe0a4b746f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:22.827514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da75fc-c326-4f83-8810-f16b5a86aac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:22.828889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da75fc-c326-4f83-8810-f16b5a86aac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:22.829466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da75fc-c326-4f83-8810-f16b5a86aac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:22.830450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da75fc-c326-4f83-8810-f16b5a86aac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:22.831216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da75fc-c326-4f83-8810-f16b5a86aac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:22.833804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da75fc-c326-4f83-8810-f16b5a86aac1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:23.025774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 921/1829] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:22 2022] POST /identity/v3/users => generated 288 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:23.027334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ba7d4-2976-48a2-8052-545c5837e9c0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:23.055032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:23.067989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ba7d4-2976-48a2-8052-545c5837e9c0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:23.074669 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8eef8e-b9fb-4967-b477-e1d98d24044f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:23.075844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8eef8e-b9fb-4967-b477-e1d98d24044f admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.075844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8eef8e-b9fb-4967-b477-e1d98d24044f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.076146 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8eef8e-b9fb-4967-b477-e1d98d24044f admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.077478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8eef8e-b9fb-4967-b477-e1d98d24044f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:23.079398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8eef8e-b9fb-4967-b477-e1d98d24044f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:23.094368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 909/1830] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:23.097899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da75fc-c326-4f83-8810-f16b5a86aac1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:23.112062 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da75fc-c326-4f83-8810-f16b5a86aac1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:23.122146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced5c20-ac99-4779-8d0c-97fbe177a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:23.122146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced5c20-ac99-4779-8d0c-97fbe177a96f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.122146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced5c20-ac99-4779-8d0c-97fbe177a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.122884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced5c20-ac99-4779-8d0c-97fbe177a96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.124089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced5c20-ac99-4779-8d0c-97fbe177a96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:23.125720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced5c20-ac99-4779-8d0c-97fbe177a96f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:23.147037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8eef8e-b9fb-4967-b477-e1d98d24044f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:23.147593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 922/1831] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:23.160389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8eef8e-b9fb-4967-b477-e1d98d24044f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:23.166316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b9c91-512a-47ee-b35f-383b11717416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:23.167647 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b9c91-512a-47ee-b35f-383b11717416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.167934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b9c91-512a-47ee-b35f-383b11717416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.168382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b9c91-512a-47ee-b35f-383b11717416 admin admin] PATH_INFO: `/v3/projects/f6df36775ca84a2780ba98fd7bce12c5/users/aae42b7920694b72b6c9c836bea17c61/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.188170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b9c91-512a-47ee-b35f-383b11717416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6df36775ca84a2780ba98fd7bce12c5, user_id=aae42b7920694b72b6c9c836bea17c61, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:23.192305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b9c91-512a-47ee-b35f-383b11717416 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:23.323437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced5c20-ac99-4779-8d0c-97fbe177a96f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:23.324835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 910/1832] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:23 2022] PUT /identity/v3/projects/f6df36775ca84a2780ba98fd7bce12c5/users/aae42b7920694b72b6c9c836bea17c61/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:23.327092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cced5c20-ac99-4779-8d0c-97fbe177a96f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:23.327341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40868fa6-f5d2-46b6-ae15-41d94509e682 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.327750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40868fa6-f5d2-46b6-ae15-41d94509e682 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.328139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40868fa6-f5d2-46b6-ae15-41d94509e682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.357440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced5c20-ac99-4779-8d0c-97fbe177a96f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:23.371398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-40868fa6-f5d2-46b6-ae15-41d94509e682 None None] MFA Rules not processed for user `aae42b7920694b72b6c9c836bea17c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:23.376479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-40868fa6-f5d2-46b6-ae15-41d94509e682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:23.521344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 911/1833] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:23.525237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40868fa6-f5d2-46b6-ae15-41d94509e682 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:23.526936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:23.527714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.527995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.528268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] PATH_INFO: `/v3/projects/2eedaeb1403d496da5abbba1808deea2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.530668 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eedaeb1403d496da5abbba1808deea2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:23.535172 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:23.541907 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40868fa6-f5d2-46b6-ae15-41d94509e682 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:23.549382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd793cbb-8cbb-4b31-9d8b-1de3f50c2587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:23.550114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd793cbb-8cbb-4b31-9d8b-1de3f50c2587 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.550779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd793cbb-8cbb-4b31-9d8b-1de3f50c2587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.551985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd793cbb-8cbb-4b31-9d8b-1de3f50c2587 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.552898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd793cbb-8cbb-4b31-9d8b-1de3f50c2587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:23.564678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd793cbb-8cbb-4b31-9d8b-1de3f50c2587 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:23.571686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:23.682610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 912/1834] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:23 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:23.686442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd793cbb-8cbb-4b31-9d8b-1de3f50c2587 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:23.736585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eedaeb1403d496da5abbba1808deea2', 'request_id': 'req-c6c10cbd-db09-4c57-a8b3-55018f274dfc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:23.745185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eedaeb1403d496da5abbba1808deea2', 'request_id': 'req-c6c10cbd-db09-4c57-a8b3-55018f274dfc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:23.824722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd793cbb-8cbb-4b31-9d8b-1de3f50c2587 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:23.849585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eedaeb1403d496da5abbba1808deea2', 'request_id': 'req-c6c10cbd-db09-4c57-a8b3-55018f274dfc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:23.863530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] The token cache is being invalidate because project 2eedaeb1403d496da5abbba1808deea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:23.864066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:23.864895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c6c10cbd-db09-4c57-a8b3-55018f274dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:23.866514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 923/1835] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:23 2022] DELETE /identity/v3/projects/2eedaeb1403d496da5abbba1808deea2 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:23.871564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf55cf2-78d3-40c0-a7a0-9ed91b6b3cd6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.872361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf55cf2-78d3-40c0-a7a0-9ed91b6b3cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.872644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf55cf2-78d3-40c0-a7a0-9ed91b6b3cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.873112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:23.906569 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a86a-2376-4aaa-b912-18d2ae07b776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:23.908016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a86a-2376-4aaa-b912-18d2ae07b776 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.908359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a86a-2376-4aaa-b912-18d2ae07b776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.908698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7a86a-2376-4aaa-b912-18d2ae07b776 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.909526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7a86a-2376-4aaa-b912-18d2ae07b776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:23.911736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7a86a-2376-4aaa-b912-18d2ae07b776 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:23.930554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5cf55cf2-78d3-40c0-a7a0-9ed91b6b3cd6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:23.931904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5cf55cf2-78d3-40c0-a7a0-9ed91b6b3cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:23.941045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 924/1836] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:23.949794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b2dc-9dbb-4089-8f2c-22803d6fb1b6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:23.949794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b2dc-9dbb-4089-8f2c-22803d6fb1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:23.949794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42b2dc-9dbb-4089-8f2c-22803d6fb1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:23.967915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1f42b2dc-9dbb-4089-8f2c-22803d6fb1b6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:23.969723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1f42b2dc-9dbb-4089-8f2c-22803d6fb1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:23.978949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 925/1837] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:23.989888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42b2dc-9dbb-4089-8f2c-22803d6fb1b6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:23.994496 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42b2dc-9dbb-4089-8f2c-22803d6fb1b6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.000023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea4cc8-45e0-4d8d-8838-6a83de68eda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.001000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea4cc8-45e0-4d8d-8838-6a83de68eda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.001347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea4cc8-45e0-4d8d-8838-6a83de68eda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.001634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea4cc8-45e0-4d8d-8838-6a83de68eda3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.002379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea4cc8-45e0-4d8d-8838-6a83de68eda3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:24.002721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea4cc8-45e0-4d8d-8838-6a83de68eda3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.004401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea4cc8-45e0-4d8d-8838-6a83de68eda3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.019132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 926/1838] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:25:24.022646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea4cc8-45e0-4d8d-8838-6a83de68eda3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.036328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea4cc8-45e0-4d8d-8838-6a83de68eda3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.041661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07de5a-5080-472c-99bd-c15d9d2d3463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.042596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07de5a-5080-472c-99bd-c15d9d2d3463 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.042884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07de5a-5080-472c-99bd-c15d9d2d3463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.043170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07de5a-5080-472c-99bd-c15d9d2d3463 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.043867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07de5a-5080-472c-99bd-c15d9d2d3463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.045879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07de5a-5080-472c-99bd-c15d9d2d3463 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.067092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 913/1839] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:23 2022] POST /identity/v3/users => generated 289 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:24.072261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a86a-2376-4aaa-b912-18d2ae07b776 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.073284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-edb7a86a-2376-4aaa-b912-18d2ae07b776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:24.086151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7a86a-2376-4aaa-b912-18d2ae07b776 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.091928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae55e0-a4e3-426c-bf5e-3bc3bc5e1742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.092670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae55e0-a4e3-426c-bf5e-3bc3bc5e1742 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.093123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae55e0-a4e3-426c-bf5e-3bc3bc5e1742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.094092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae55e0-a4e3-426c-bf5e-3bc3bc5e1742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.095032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae55e0-a4e3-426c-bf5e-3bc3bc5e1742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.096893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae55e0-a4e3-426c-bf5e-3bc3bc5e1742 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.108543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 914/1840] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:24.113426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae55e0-a4e3-426c-bf5e-3bc3bc5e1742 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.124330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae55e0-a4e3-426c-bf5e-3bc3bc5e1742 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.130165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40cf7d-b35d-4cda-9f8c-2f9f9667ab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.130868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40cf7d-b35d-4cda-9f8c-2f9f9667ab74 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.131149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40cf7d-b35d-4cda-9f8c-2f9f9667ab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.131429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40cf7d-b35d-4cda-9f8c-2f9f9667ab74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.132157 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40cf7d-b35d-4cda-9f8c-2f9f9667ab74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.133749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40cf7d-b35d-4cda-9f8c-2f9f9667ab74 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.150841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 915/1841] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:24.155143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40cf7d-b35d-4cda-9f8c-2f9f9667ab74 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.169188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40cf7d-b35d-4cda-9f8c-2f9f9667ab74 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.177106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491eebe-c235-43c2-bd3b-8d0e4f944c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.178144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491eebe-c235-43c2-bd3b-8d0e4f944c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.178538 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491eebe-c235-43c2-bd3b-8d0e4f944c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.179750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491eebe-c235-43c2-bd3b-8d0e4f944c1d admin admin] PATH_INFO: `/v3/projects/1152e63c911c45debe65efd8bb40944a/users/6fd982f28feb4a7d86089adccc0b6c2b/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.198465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491eebe-c235-43c2-bd3b-8d0e4f944c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1152e63c911c45debe65efd8bb40944a, user_id=6fd982f28feb4a7d86089adccc0b6c2b, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.199658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 927/1842] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:24.204000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07de5a-5080-472c-99bd-c15d9d2d3463 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.204478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491eebe-c235-43c2-bd3b-8d0e4f944c1d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.269506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07de5a-5080-472c-99bd-c15d9d2d3463 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.274366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1971b-e301-4396-84b0-5bb41d3ea97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.275083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1971b-e301-4396-84b0-5bb41d3ea97c admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.275362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1971b-e301-4396-84b0-5bb41d3ea97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.278913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1971b-e301-4396-84b0-5bb41d3ea97c admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.278913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1971b-e301-4396-84b0-5bb41d3ea97c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.278913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1971b-e301-4396-84b0-5bb41d3ea97c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.312766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 916/1843] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:24 2022] PUT /identity/v3/projects/1152e63c911c45debe65efd8bb40944a/users/6fd982f28feb4a7d86089adccc0b6c2b/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:24.335918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202b5f6-e7a8-47a5-8392-587475a74576 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.336399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202b5f6-e7a8-47a5-8392-587475a74576 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.339081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202b5f6-e7a8-47a5-8392-587475a74576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.366955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 928/1844] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:24.373722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1971b-e301-4396-84b0-5bb41d3ea97c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.377752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f202b5f6-e7a8-47a5-8392-587475a74576 None None] MFA Rules not processed for user `6fd982f28feb4a7d86089adccc0b6c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:24.378317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f202b5f6-e7a8-47a5-8392-587475a74576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:24.392958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1971b-e301-4396-84b0-5bb41d3ea97c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.428211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:24.459621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 917/1845] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:24.460581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202b5f6-e7a8-47a5-8392-587475a74576 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.478863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62834157-aa33-497c-aa5a-5665d7b6856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.479544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62834157-aa33-497c-aa5a-5665d7b6856f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.480052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62834157-aa33-497c-aa5a-5665d7b6856f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.480196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62834157-aa33-497c-aa5a-5665d7b6856f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.480940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62834157-aa33-497c-aa5a-5665d7b6856f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.482640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62834157-aa33-497c-aa5a-5665d7b6856f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.494901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 929/1846] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:24.500713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62834157-aa33-497c-aa5a-5665d7b6856f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.510783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62834157-aa33-497c-aa5a-5665d7b6856f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.515908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68f48b-9d7a-4dee-906c-7d24689ddeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.516788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68f48b-9d7a-4dee-906c-7d24689ddeaf admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.517148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68f48b-9d7a-4dee-906c-7d24689ddeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.517503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68f48b-9d7a-4dee-906c-7d24689ddeaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.518394 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68f48b-9d7a-4dee-906c-7d24689ddeaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.520205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68f48b-9d7a-4dee-906c-7d24689ddeaf admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.532042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 930/1847] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:24.539138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202b5f6-e7a8-47a5-8392-587475a74576 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.540693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68f48b-9d7a-4dee-906c-7d24689ddeaf admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.548724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a1198-c8ca-4cff-ade6-de0aeae95e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.549681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a1198-c8ca-4cff-ade6-de0aeae95e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.549995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a1198-c8ca-4cff-ade6-de0aeae95e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.551026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a1198-c8ca-4cff-ade6-de0aeae95e3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.551785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a1198-c8ca-4cff-ade6-de0aeae95e3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.553762 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a1198-c8ca-4cff-ade6-de0aeae95e3c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.556635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68f48b-9d7a-4dee-906c-7d24689ddeaf admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.561945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d43e-d9b1-4936-a41a-df16825a0bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:24.563791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d43e-d9b1-4936-a41a-df16825a0bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.564141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d43e-d9b1-4936-a41a-df16825a0bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.564441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d43e-d9b1-4936-a41a-df16825a0bbe admin admin] PATH_INFO: `/v3/projects/9f556df7a95041279685098584415ed1/users/7caa126f7eab4eef8db03374e82a6ab9/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.583186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94d43e-d9b1-4936-a41a-df16825a0bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f556df7a95041279685098584415ed1, user_id=7caa126f7eab4eef8db03374e82a6ab9, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:24.586520 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94d43e-d9b1-4936-a41a-df16825a0bbe admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:24.605985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:24.806463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 918/1848] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/projects => generated 379 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:24.807054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a1198-c8ca-4cff-ade6-de0aeae95e3c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:24.907345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 931/1849] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:24 2022] PUT /identity/v3/projects/9f556df7a95041279685098584415ed1/users/7caa126f7eab4eef8db03374e82a6ab9/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:24.912996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5018-922e-4a78-be5a-b01dcbda1032 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:24.913553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5018-922e-4a78-be5a-b01dcbda1032 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:24.913983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d5018-922e-4a78-be5a-b01dcbda1032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:24.931744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a1198-c8ca-4cff-ade6-de0aeae95e3c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:24.946782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0d9d5018-922e-4a78-be5a-b01dcbda1032 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:24.948012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0d9d5018-922e-4a78-be5a-b01dcbda1032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:25.005335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5a036-6ece-48a6-8476-2048f59c0d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:25.006388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5a036-6ece-48a6-8476-2048f59c0d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:25.006871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5a036-6ece-48a6-8476-2048f59c0d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:25.007336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5a036-6ece-48a6-8476-2048f59c0d53 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:25.008379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5a036-6ece-48a6-8476-2048f59c0d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:25.010580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5a036-6ece-48a6-8476-2048f59c0d53 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:25.038791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 932/1850] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:25.049847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d5018-922e-4a78-be5a-b01dcbda1032 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:25.052218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0d9d5018-922e-4a78-be5a-b01dcbda1032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:25.163997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 919/1851] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:24 2022] POST /identity/v3/users => generated 296 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:25.164663 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d5018-922e-4a78-be5a-b01dcbda1032 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:25.168234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5a036-6ece-48a6-8476-2048f59c0d53 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:25.170265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96682077-330c-440e-9a3b-1e637c5cf0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:25.171039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96682077-330c-440e-9a3b-1e637c5cf0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:25.171729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96682077-330c-440e-9a3b-1e637c5cf0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:25.172426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96682077-330c-440e-9a3b-1e637c5cf0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:25.179425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5a036-6ece-48a6-8476-2048f59c0d53 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:25.184338 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96682077-330c-440e-9a3b-1e637c5cf0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:25.185597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91726578-6100-4c70-90a9-a48767462c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:25.186136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96682077-330c-440e-9a3b-1e637c5cf0a5 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:25.186260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91726578-6100-4c70-90a9-a48767462c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:25.186596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91726578-6100-4c70-90a9-a48767462c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:25.186878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91726578-6100-4c70-90a9-a48767462c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:25.188360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91726578-6100-4c70-90a9-a48767462c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:25.190196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91726578-6100-4c70-90a9-a48767462c09 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:25.204274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 920/1852] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:25.217639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 933/1853] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:25:25.223450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91726578-6100-4c70-90a9-a48767462c09 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:25.257604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91726578-6100-4c70-90a9-a48767462c09 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:25.257604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f808a4b-0984-4f91-9915-48723c30c961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:25.257604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808a4b-0984-4f91-9915-48723c30c961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:25.257604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808a4b-0984-4f91-9915-48723c30c961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:25.258058 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f808a4b-0984-4f91-9915-48723c30c961 admin admin] PATH_INFO: `/v3/projects/13f6552749c647ec8210783beb2f7161/users/ef93a7f4fe4e421bac86bf096c6cddc8/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:25.294151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f808a4b-0984-4f91-9915-48723c30c961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f6552749c647ec8210783beb2f7161, user_id=ef93a7f4fe4e421bac86bf096c6cddc8, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:25.298043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f808a4b-0984-4f91-9915-48723c30c961 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:25.431684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 921/1854] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:25 2022] PUT /identity/v3/projects/13f6552749c647ec8210783beb2f7161/users/ef93a7f4fe4e421bac86bf096c6cddc8/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:25.435948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96682077-330c-440e-9a3b-1e637c5cf0a5 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:25.454766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96682077-330c-440e-9a3b-1e637c5cf0a5 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:25.565703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29416582-e4fe-43f6-ab8f-6e344119ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:25.566833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29416582-e4fe-43f6-ab8f-6e344119ba0d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:25.567338 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29416582-e4fe-43f6-ab8f-6e344119ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:25.567795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29416582-e4fe-43f6-ab8f-6e344119ba0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:25.569049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29416582-e4fe-43f6-ab8f-6e344119ba0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:25.571965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29416582-e4fe-43f6-ab8f-6e344119ba0d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:25.590516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 934/1855] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:25.595251 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f808a4b-0984-4f91-9915-48723c30c961 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:25.622815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f808a4b-0984-4f91-9915-48723c30c961 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:25.642840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f2033-c618-445c-b266-a70c156e652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:25.652341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f2033-c618-445c-b266-a70c156e652f admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:25.652341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f2033-c618-445c-b266-a70c156e652f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:25.652341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f2033-c618-445c-b266-a70c156e652f admin admin] PATH_INFO: `/v3/projects/13f6552749c647ec8210783beb2f7161/users/ef93a7f4fe4e421bac86bf096c6cddc8/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:25.652341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f2033-c618-445c-b266-a70c156e652f admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f6552749c647ec8210783beb2f7161, user_id=ef93a7f4fe4e421bac86bf096c6cddc8, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:25.652341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f2033-c618-445c-b266-a70c156e652f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:25.683134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 922/1856] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:25 2022] PUT /identity/v3/projects/13f6552749c647ec8210783beb2f7161/users/ef93a7f4fe4e421bac86bf096c6cddc8/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:25.687938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee37cb-0a23-4e4f-9bca-89ed47f8b3e7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:25.688572 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee37cb-0a23-4e4f-9bca-89ed47f8b3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:25.688572 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee37cb-0a23-4e4f-9bca-89ed47f8b3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:25.742747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-29ee37cb-0a23-4e4f-9bca-89ed47f8b3e7 None None] MFA Rules not processed for user `ef93a7f4fe4e421bac86bf096c6cddc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:25.744134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-29ee37cb-0a23-4e4f-9bca-89ed47f8b3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:25.875278 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 935/1857] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:25.877464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f2033-c618-445c-b266-a70c156e652f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:25.884098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-907f2033-c618-445c-b266-a70c156e652f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:25.965773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:25.981707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f2033-c618-445c-b266-a70c156e652f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:25.991802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12930f9-6d5f-415f-9140-a2541148c23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:25.992851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12930f9-6d5f-415f-9140-a2541148c23e service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:25.993293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12930f9-6d5f-415f-9140-a2541148c23e service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:25.993878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12930f9-6d5f-415f-9140-a2541148c23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:26.000671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:26.007477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12930f9-6d5f-415f-9140-a2541148c23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:26.010018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12930f9-6d5f-415f-9140-a2541148c23e service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:26.022210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 923/1858] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:26.171246 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee37cb-0a23-4e4f-9bca-89ed47f8b3e7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:26.174437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-29ee37cb-0a23-4e4f-9bca-89ed47f8b3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:26.189355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee37cb-0a23-4e4f-9bca-89ed47f8b3e7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:26.194516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0580f-69cb-48d0-933c-330b87a4996b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:26.195286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0580f-69cb-48d0-933c-330b87a4996b service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:26.195641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0580f-69cb-48d0-933c-330b87a4996b service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:26.196007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0580f-69cb-48d0-933c-330b87a4996b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:26.198027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7ca0580f-69cb-48d0-933c-330b87a4996b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:26.292742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0580f-69cb-48d0-933c-330b87a4996b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:26.294208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0580f-69cb-48d0-933c-330b87a4996b service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:26.305922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 936/1859] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:27.127866 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf81c6-9eda-4b93-b20b-3ac3255a78b1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:27.128508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf81c6-9eda-4b93-b20b-3ac3255a78b1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:27.129048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf81c6-9eda-4b93-b20b-3ac3255a78b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:27.167300 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-dadf81c6-9eda-4b93-b20b-3ac3255a78b1 None None] MFA Rules not processed for user `7caa126f7eab4eef8db03374e82a6ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:27.169757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dadf81c6-9eda-4b93-b20b-3ac3255a78b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:27.282575 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 924/1860] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:27.294637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0580f-69cb-48d0-933c-330b87a4996b service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:27.304221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0580f-69cb-48d0-933c-330b87a4996b service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:27.312913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58260f64-69b6-4005-9887-5162069d92b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:27.313815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58260f64-69b6-4005-9887-5162069d92b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:27.314383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58260f64-69b6-4005-9887-5162069d92b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:27.314654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58260f64-69b6-4005-9887-5162069d92b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:27.330033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58260f64-69b6-4005-9887-5162069d92b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:27.331994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58260f64-69b6-4005-9887-5162069d92b0 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:27.337220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:27.343821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 937/1861] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:31.384350 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f170fa-b0ba-4870-a20b-a136232bf4d0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:31.384783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f170fa-b0ba-4870-a20b-a136232bf4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:31.384783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f170fa-b0ba-4870-a20b-a136232bf4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:31.426559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-05f170fa-b0ba-4870-a20b-a136232bf4d0 None None] MFA Rules not processed for user `7caa126f7eab4eef8db03374e82a6ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:31.428763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-05f170fa-b0ba-4870-a20b-a136232bf4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:31.447257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 925/1862] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:33.200799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58260f64-69b6-4005-9887-5162069d92b0 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:33.379269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58260f64-69b6-4005-9887-5162069d92b0 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:33.384388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:33.385286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:33.385570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:33.385854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] PATH_INFO: `/v3/users/aae42b7920694b72b6c9c836bea17c61` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:33.387164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae42b7920694b72b6c9c836bea17c61)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:33.389142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:33.684609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae42b7920694b72b6c9c836bea17c61'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:33.706024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae42b7920694b72b6c9c836bea17c61'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:33.706024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae42b7920694b72b6c9c836bea17c61'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:33.706392 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae42b7920694b72b6c9c836bea17c61'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:33.707590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f170fa-b0ba-4870-a20b-a136232bf4d0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:33.709303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-05f170fa-b0ba-4870-a20b-a136232bf4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:33.724580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f170fa-b0ba-4870-a20b-a136232bf4d0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:33.729681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:33.730653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:33.731008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:33.731360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] PATH_INFO: `/v3/users/4ad0bd03fd2e4d19bbce4c31096d69c8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:33.732761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad0bd03fd2e4d19bbce4c31096d69c8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:33.734962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:33.877349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ad7e0646-c7ec-4d59-9aa1-506e8af7f30b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae42b7920694b72b6c9c836bea17c61'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:33.914781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 938/1863] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:33 2022] DELETE /identity/v3/users/aae42b7920694b72b6c9c836bea17c61 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:33.920245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e62443-8d4a-43d0-aff8-7211256de5d0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:33.920647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e62443-8d4a-43d0-aff8-7211256de5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:33.920972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e62443-8d4a-43d0-aff8-7211256de5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:33.944871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-33e62443-8d4a-43d0-aff8-7211256de5d0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:33.944871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-33e62443-8d4a-43d0-aff8-7211256de5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:34.014020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:34.023844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 939/1864] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:34.035829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e62443-8d4a-43d0-aff8-7211256de5d0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:34.037498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-33e62443-8d4a-43d0-aff8-7211256de5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:34.125164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e62443-8d4a-43d0-aff8-7211256de5d0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:34.139682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf708b-5eda-4ef8-b6f9-6377dfa8aa8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:34.139682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf708b-5eda-4ef8-b6f9-6377dfa8aa8f service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:34.139682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf708b-5eda-4ef8-b6f9-6377dfa8aa8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:34.139682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf708b-5eda-4ef8-b6f9-6377dfa8aa8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:34.148092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf708b-5eda-4ef8-b6f9-6377dfa8aa8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:34.149940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf708b-5eda-4ef8-b6f9-6377dfa8aa8f service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:34.163345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 940/1865] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:25:34.257448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad0bd03fd2e4d19bbce4c31096d69c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:34.258607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad0bd03fd2e4d19bbce4c31096d69c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:34.278497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad0bd03fd2e4d19bbce4c31096d69c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:34.279527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad0bd03fd2e4d19bbce4c31096d69c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:34.392571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad0bd03fd2e4d19bbce4c31096d69c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:34.430268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 926/1866] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:33 2022] DELETE /identity/v3/users/4ad0bd03fd2e4d19bbce4c31096d69c8 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:34.432454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3a40d-1137-439b-9226-e2b989fc7049 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:34.433265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3a40d-1137-439b-9226-e2b989fc7049 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:34.433439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3a40d-1137-439b-9226-e2b989fc7049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:34.463035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-1cc3a40d-1137-439b-9226-e2b989fc7049 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:34.465005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1cc3a40d-1137-439b-9226-e2b989fc7049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:34.588559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 941/1867] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:34.597981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:34.601166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:34.695616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:34.722040 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12f285-3294-458e-af87-e991f2351025 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:34.729279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb83e0-f9c6-40a1-a520-f3ae542a71f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:34.729995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb83e0-f9c6-40a1-a520-f3ae542a71f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:34.730307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb83e0-f9c6-40a1-a520-f3ae542a71f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:34.730930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb83e0-f9c6-40a1-a520-f3ae542a71f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:34.742367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb83e0-f9c6-40a1-a520-f3ae542a71f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:34.743888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb83e0-f9c6-40a1-a520-f3ae542a71f0 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:34.755487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 927/1868] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:35.375031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3a40d-1137-439b-9226-e2b989fc7049 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:35.376662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1cc3a40d-1137-439b-9226-e2b989fc7049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:35.389212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3a40d-1137-439b-9226-e2b989fc7049 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:35.395141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:35.396148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:35.403721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:35.403721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] PATH_INFO: `/v3/projects/f6df36775ca84a2780ba98fd7bce12c5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:35.403721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6df36775ca84a2780ba98fd7bce12c5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:35.403721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:35.494197 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb83e0-f9c6-40a1-a520-f3ae542a71f0 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:35.496249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-00eb83e0-f9c6-40a1-a520-f3ae542a71f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:35.512942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb83e0-f9c6-40a1-a520-f3ae542a71f0 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:35.521431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:35.522680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:35.523116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:35.523585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] PATH_INFO: `/v3/projects/8484f3ad67724bc2bd59dd170e23d7ee` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:35.526768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8484f3ad67724bc2bd59dd170e23d7ee)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:35.530073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:35.629516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6df36775ca84a2780ba98fd7bce12c5', 'request_id': 'req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.637119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6df36775ca84a2780ba98fd7bce12c5', 'request_id': 'req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.793111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8484f3ad67724bc2bd59dd170e23d7ee', 'request_id': 'req-1a05def0-b784-44d1-9b56-699a0ce7690c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.798956 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8484f3ad67724bc2bd59dd170e23d7ee', 'request_id': 'req-1a05def0-b784-44d1-9b56-699a0ce7690c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.802762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6df36775ca84a2780ba98fd7bce12c5', 'request_id': 'req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.817054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] The token cache is being invalidate because project f6df36775ca84a2780ba98fd7bce12c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:35.817598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.818274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.819875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 942/1869] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:35 2022] DELETE /identity/v3/projects/f6df36775ca84a2780ba98fd7bce12c5 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:35.823981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:35.825803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:35.860115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:35.896756 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8484f3ad67724bc2bd59dd170e23d7ee', 'request_id': 'req-1a05def0-b784-44d1-9b56-699a0ce7690c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.898318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b706a-ee2d-4caf-a8b1-7405e85dafb3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:35.913731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] The token cache is being invalidate because project 8484f3ad67724bc2bd59dd170e23d7ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:35.913731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.914571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:35.919621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 928/1870] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:35 2022] DELETE /identity/v3/projects/8484f3ad67724bc2bd59dd170e23d7ee => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:35.920940 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:35.924179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:35.985083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:35.986082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:35.986371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:35.986707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] PATH_INFO: `/v3/users/6fd982f28feb4a7d86089adccc0b6c2b` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:35.987925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd982f28feb4a7d86089adccc0b6c2b)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:35.990160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:36.045070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05def0-b784-44d1-9b56-699a0ce7690c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:36.055129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:36.057268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:36.057853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:36.058386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] PATH_INFO: `/v3/users/411a1b7163464f0fa4efe13e4dfe5c11` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:36.060871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=411a1b7163464f0fa4efe13e4dfe5c11)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:36.065516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:36.314772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd982f28feb4a7d86089adccc0b6c2b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.322480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd982f28feb4a7d86089adccc0b6c2b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.323212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd982f28feb4a7d86089adccc0b6c2b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.323843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd982f28feb4a7d86089adccc0b6c2b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.365812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '411a1b7163464f0fa4efe13e4dfe5c11'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.366636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '411a1b7163464f0fa4efe13e4dfe5c11'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.382844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '411a1b7163464f0fa4efe13e4dfe5c11'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.383894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '411a1b7163464f0fa4efe13e4dfe5c11'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.390649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:36.419415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd982f28feb4a7d86089adccc0b6c2b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.447581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 943/1871] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:35 2022] DELETE /identity/v3/users/6fd982f28feb4a7d86089adccc0b6c2b => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:36.539836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '411a1b7163464f0fa4efe13e4dfe5c11'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:36.576703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 929/1872] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:35 2022] DELETE /identity/v3/users/411a1b7163464f0fa4efe13e4dfe5c11 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:37.136156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:37.138206 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:37.171452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:37.174288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:37.250576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e71071-59e0-4bf5-a2ac-2986ab2fe39a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:37.276219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6e560-5dcf-4361-a7a2-f1a57bffbe36 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:37.279824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:37.288021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:37.290082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:37.290973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:37.291784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] PATH_INFO: `/v3/projects/1152e63c911c45debe65efd8bb40944a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:37.294495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1152e63c911c45debe65efd8bb40944a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:37.299051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:37.328875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:37.329791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:37.330080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:37.330363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] PATH_INFO: `/v3/projects/345a24ac225e42eab20c5f5601d7509c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:37.331853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] RBAC: Authorizing `identity:delete_project(project_id=345a24ac225e42eab20c5f5601d7509c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:37.333820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:37.457719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1152e63c911c45debe65efd8bb40944a', 'request_id': 'req-3044c211-c204-45f9-b7be-4a06c517adf0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.463111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1152e63c911c45debe65efd8bb40944a', 'request_id': 'req-3044c211-c204-45f9-b7be-4a06c517adf0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.494465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345a24ac225e42eab20c5f5601d7509c', 'request_id': 'req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.501422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345a24ac225e42eab20c5f5601d7509c', 'request_id': 'req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.550008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1152e63c911c45debe65efd8bb40944a', 'request_id': 'req-3044c211-c204-45f9-b7be-4a06c517adf0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.564489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] The token cache is being invalidate because project 1152e63c911c45debe65efd8bb40944a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:37.564885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.565738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.567725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 930/1873] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:37 2022] DELETE /identity/v3/projects/1152e63c911c45debe65efd8bb40944a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:37.572066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:37.573539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:37.677084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044c211-c204-45f9-b7be-4a06c517adf0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:37.750736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345a24ac225e42eab20c5f5601d7509c', 'request_id': 'req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.777165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] The token cache is being invalidate because project 345a24ac225e42eab20c5f5601d7509c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:37.777721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.780076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6efc0ce3-78a8-4075-9cb7-8ecdae59c74b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:37.782918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 944/1874] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:37 2022] DELETE /identity/v3/projects/345a24ac225e42eab20c5f5601d7509c => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:37.788768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd77f5-6c70-45da-ab7a-105021ca0e0a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:37.789432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd77f5-6c70-45da-ab7a-105021ca0e0a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:37.790160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd77f5-6c70-45da-ab7a-105021ca0e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:37.804370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:37.805586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:37.805972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:37.806357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] PATH_INFO: `/v3/users/ef93a7f4fe4e421bac86bf096c6cddc8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:37.807894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef93a7f4fe4e421bac86bf096c6cddc8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:37.816614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:37.827489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-95bd77f5-6c70-45da-ab7a-105021ca0e0a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:37.829089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-95bd77f5-6c70-45da-ab7a-105021ca0e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:37.847959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 945/1875] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:37.849526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffa6ca-66b7-47b8-be3a-b5a3c4963f3e None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:37.849825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffa6ca-66b7-47b8-be3a-b5a3c4963f3e None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:37.850111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffa6ca-66b7-47b8-be3a-b5a3c4963f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:37.865302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:37.875176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-83ffa6ca-66b7-47b8-be3a-b5a3c4963f3e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:37.876153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-83ffa6ca-66b7-47b8-be3a-b5a3c4963f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:37.890529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 946/1876] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:37.891778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffa6ca-66b7-47b8-be3a-b5a3c4963f3e None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:37.903536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffa6ca-66b7-47b8-be3a-b5a3c4963f3e None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:37.909525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fa021-15ea-497f-8edc-62081b2b47a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:37.910219 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa021-15ea-497f-8edc-62081b2b47a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:37.910575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa021-15ea-497f-8edc-62081b2b47a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:37.911262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fa021-15ea-497f-8edc-62081b2b47a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:37.912028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fa021-15ea-497f-8edc-62081b2b47a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:37.912518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fa021-15ea-497f-8edc-62081b2b47a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:37.914158 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fa021-15ea-497f-8edc-62081b2b47a8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:37.934464 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 947/1877] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:37.941460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fa021-15ea-497f-8edc-62081b2b47a8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:37.955988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fa021-15ea-497f-8edc-62081b2b47a8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:37.972473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce2a49-1770-4953-86cf-69c8c321649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:37.972473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce2a49-1770-4953-86cf-69c8c321649b admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:37.972714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce2a49-1770-4953-86cf-69c8c321649b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:37.972950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce2a49-1770-4953-86cf-69c8c321649b admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:37.975079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce2a49-1770-4953-86cf-69c8c321649b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:37.977403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce2a49-1770-4953-86cf-69c8c321649b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:38.135436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef93a7f4fe4e421bac86bf096c6cddc8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:38.136140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef93a7f4fe4e421bac86bf096c6cddc8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:38.137489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 948/1878] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:37 2022] POST /identity/v3/projects => generated 369 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:38.148532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce2a49-1770-4953-86cf-69c8c321649b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:38.150781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-73ce2a49-1770-4953-86cf-69c8c321649b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:38.157206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef93a7f4fe4e421bac86bf096c6cddc8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:38.157975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef93a7f4fe4e421bac86bf096c6cddc8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:38.232005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce2a49-1770-4953-86cf-69c8c321649b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:38.240675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10304f-347f-44c9-beac-3eb1c515570a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:38.242494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10304f-347f-44c9-beac-3eb1c515570a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:38.242924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10304f-347f-44c9-beac-3eb1c515570a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:38.243308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10304f-347f-44c9-beac-3eb1c515570a admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:38.244170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10304f-347f-44c9-beac-3eb1c515570a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:38.250550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10304f-347f-44c9-beac-3eb1c515570a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:38.290968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef93a7f4fe4e421bac86bf096c6cddc8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:38.339252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 931/1879] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:37 2022] DELETE /identity/v3/users/ef93a7f4fe4e421bac86bf096c6cddc8 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:38.385841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 949/1880] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:38 2022] POST /identity/v3/users => generated 285 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:38.390471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:38.522996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0845fcb-b669-4a34-9211-a06028cfba74 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:38.529000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7c4d3-8bc7-43bb-bb80-e8968d5c8378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:38.529697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7c4d3-8bc7-43bb-bb80-e8968d5c8378 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:38.529983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7c4d3-8bc7-43bb-bb80-e8968d5c8378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:38.530261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7c4d3-8bc7-43bb-bb80-e8968d5c8378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:38.532063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7c4d3-8bc7-43bb-bb80-e8968d5c8378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:38.533699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7c4d3-8bc7-43bb-bb80-e8968d5c8378 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:38.547136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 932/1881] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:38.551650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10304f-347f-44c9-beac-3eb1c515570a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:38.563349 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10304f-347f-44c9-beac-3eb1c515570a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:38.569796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3952a3-73f5-4004-9156-2874831d1a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:38.570516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3952a3-73f5-4004-9156-2874831d1a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:38.570804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3952a3-73f5-4004-9156-2874831d1a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:38.571089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3952a3-73f5-4004-9156-2874831d1a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:38.572205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3952a3-73f5-4004-9156-2874831d1a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:38.573853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3952a3-73f5-4004-9156-2874831d1a3c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:38.588068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 950/1882] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:38.588068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:38.589897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7c4d3-8bc7-43bb-bb80-e8968d5c8378 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:38.605179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7c4d3-8bc7-43bb-bb80-e8968d5c8378 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:38.611649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a3da7-296f-4c95-ac0f-b87a5846f19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:38.616153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a3da7-296f-4c95-ac0f-b87a5846f19a admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:38.616153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a3da7-296f-4c95-ac0f-b87a5846f19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:38.616153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a3da7-296f-4c95-ac0f-b87a5846f19a admin admin] PATH_INFO: `/v3/projects/6f769e516f5041798aada26d25ac54c5/users/099ac957cbec4602ab71504aedb62d6d/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:38.642163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a3da7-296f-4c95-ac0f-b87a5846f19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f769e516f5041798aada26d25ac54c5, user_id=099ac957cbec4602ab71504aedb62d6d, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:38.646529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a3da7-296f-4c95-ac0f-b87a5846f19a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:38.848066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 933/1883] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:38 2022] PUT /identity/v3/projects/6f769e516f5041798aada26d25ac54c5/users/099ac957cbec4602ab71504aedb62d6d/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:38.859410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7be33-df0f-4e69-83a9-d056b5d29699 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:38.859916 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7be33-df0f-4e69-83a9-d056b5d29699 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:38.860490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7be33-df0f-4e69-83a9-d056b5d29699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:38.892166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-71c7be33-df0f-4e69-83a9-d056b5d29699 None None] MFA Rules not processed for user `099ac957cbec4602ab71504aedb62d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:38.893400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-71c7be33-df0f-4e69-83a9-d056b5d29699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:38.977431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 951/1884] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:38.979286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a3da7-296f-4c95-ac0f-b87a5846f19a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.079917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a3da7-296f-4c95-ac0f-b87a5846f19a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:39.085424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35514526-2e71-4709-9e32-4e7b9b7ade55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:39.086112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35514526-2e71-4709-9e32-4e7b9b7ade55 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.086396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35514526-2e71-4709-9e32-4e7b9b7ade55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.086739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35514526-2e71-4709-9e32-4e7b9b7ade55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.087500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35514526-2e71-4709-9e32-4e7b9b7ade55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:39.089952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35514526-2e71-4709-9e32-4e7b9b7ade55 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:39.168693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7be33-df0f-4e69-83a9-d056b5d29699 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.171872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-71c7be33-df0f-4e69-83a9-d056b5d29699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:39.190744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7be33-df0f-4e69-83a9-d056b5d29699 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:39.198716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:39.199601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.199887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.200167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] PATH_INFO: `/v3/projects/13f6552749c647ec8210783beb2f7161` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.201405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f6552749c647ec8210783beb2f7161)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:39.203458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:39.253444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 934/1885] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:38 2022] POST /identity/v3/projects => generated 367 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:39.257165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35514526-2e71-4709-9e32-4e7b9b7ade55 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.327802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35514526-2e71-4709-9e32-4e7b9b7ade55 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:39.333082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7608b1-091f-49d3-a556-451e0768b854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:39.334043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7608b1-091f-49d3-a556-451e0768b854 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.334182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7608b1-091f-49d3-a556-451e0768b854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.334548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7608b1-091f-49d3-a556-451e0768b854 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.335322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7608b1-091f-49d3-a556-451e0768b854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:39.337339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7608b1-091f-49d3-a556-451e0768b854 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:39.371609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f6552749c647ec8210783beb2f7161', 'request_id': 'req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:39.379562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f6552749c647ec8210783beb2f7161', 'request_id': 'req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:39.490592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f6552749c647ec8210783beb2f7161', 'request_id': 'req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:39.493855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 935/1886] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:39 2022] POST /identity/v3/users => generated 290 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:39.498097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7608b1-091f-49d3-a556-451e0768b854 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.507366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] The token cache is being invalidate because project 13f6552749c647ec8210783beb2f7161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:39.507658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:39.508474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e35a1ebe-8aa8-4a0f-b2e7-efd66f00e41b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:39.510154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 952/1887] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:39 2022] DELETE /identity/v3/projects/13f6552749c647ec8210783beb2f7161 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:39.517055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ce73-2f16-4ba4-910c-281819c50ddb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.517448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ce73-2f16-4ba4-910c-281819c50ddb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.517728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ce73-2f16-4ba4-910c-281819c50ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.518625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7608b1-091f-49d3-a556-451e0768b854 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:39.538639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5894ce73-2f16-4ba4-910c-281819c50ddb None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:39.539872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5894ce73-2f16-4ba4-910c-281819c50ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:39.606852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:39.627240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 953/1888] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:39.632970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400402f6-3d4f-457f-a786-5e974af650cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:39.633973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400402f6-3d4f-457f-a786-5e974af650cf admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.634340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400402f6-3d4f-457f-a786-5e974af650cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.634668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400402f6-3d4f-457f-a786-5e974af650cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.635492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400402f6-3d4f-457f-a786-5e974af650cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:39.637709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa6032-602a-4a53-b4e0-3fac86b7a2c0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.638052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa6032-602a-4a53-b4e0-3fac86b7a2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.638332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa6032-602a-4a53-b4e0-3fac86b7a2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.643043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400402f6-3d4f-457f-a786-5e974af650cf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:39.664121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 936/1889] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:39.666381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400402f6-3d4f-457f-a786-5e974af650cf admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.669668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-400402f6-3d4f-457f-a786-5e974af650cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:39.690908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-4aaa6032-602a-4a53-b4e0-3fac86b7a2c0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:39.694887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4aaa6032-602a-4a53-b4e0-3fac86b7a2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:39.696771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400402f6-3d4f-457f-a786-5e974af650cf admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:39.705844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a2f8b-1c71-4da4-bc66-f47d406b706f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:39.707131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a2f8b-1c71-4da4-bc66-f47d406b706f admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.707570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a2f8b-1c71-4da4-bc66-f47d406b706f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.712050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a2f8b-1c71-4da4-bc66-f47d406b706f admin admin] PATH_INFO: `/v3/projects/b65016bbd5b5491987926accbbbe8025/users/207eb77d28fa429b91e182e223c26adc/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.722486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 954/1890] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:39.726822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa6032-602a-4a53-b4e0-3fac86b7a2c0 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.738110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a2f8b-1c71-4da4-bc66-f47d406b706f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b65016bbd5b5491987926accbbbe8025, user_id=207eb77d28fa429b91e182e223c26adc, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:39.738492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa6032-602a-4a53-b4e0-3fac86b7a2c0 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:39.741082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8a2f8b-1c71-4da4-bc66-f47d406b706f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:39.746188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc292c1f-f96a-4096-9bb2-acde350a1ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:39.748553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc292c1f-f96a-4096-9bb2-acde350a1ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.748553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc292c1f-f96a-4096-9bb2-acde350a1ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.748553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc292c1f-f96a-4096-9bb2-acde350a1ecc admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.748553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc292c1f-f96a-4096-9bb2-acde350a1ecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:39.750988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc292c1f-f96a-4096-9bb2-acde350a1ecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:39.752665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc292c1f-f96a-4096-9bb2-acde350a1ecc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:39.785569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 955/1891] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:39.787240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc292c1f-f96a-4096-9bb2-acde350a1ecc admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.802171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc292c1f-f96a-4096-9bb2-acde350a1ecc admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:39.808666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb722325-fd56-4f77-8f87-114422f39eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:39.809747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb722325-fd56-4f77-8f87-114422f39eea admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.810041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb722325-fd56-4f77-8f87-114422f39eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.810324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb722325-fd56-4f77-8f87-114422f39eea admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.811087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb722325-fd56-4f77-8f87-114422f39eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:39.814035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb722325-fd56-4f77-8f87-114422f39eea admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:39.842974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 937/1892] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:39 2022] PUT /identity/v3/projects/b65016bbd5b5491987926accbbbe8025/users/207eb77d28fa429b91e182e223c26adc/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:39.844526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a2f8b-1c71-4da4-bc66-f47d406b706f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.922848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8a2f8b-1c71-4da4-bc66-f47d406b706f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:39.935006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf911e8-cedc-4f69-a431-2b52bb790351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:39.935738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf911e8-cedc-4f69-a431-2b52bb790351 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:39.936017 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf911e8-cedc-4f69-a431-2b52bb790351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:39.936376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf911e8-cedc-4f69-a431-2b52bb790351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:39.939699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf911e8-cedc-4f69-a431-2b52bb790351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:39.941381 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf911e8-cedc-4f69-a431-2b52bb790351 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:39.957343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 938/1893] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:39.957604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 956/1894] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:39 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:39.959500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb722325-fd56-4f77-8f87-114422f39eea admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.959655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf911e8-cedc-4f69-a431-2b52bb790351 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:39.970542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf911e8-cedc-4f69-a431-2b52bb790351 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:40.050922 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68187a91-93fd-4cd8-8f20-17f0d11a8864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:40.050922 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68187a91-93fd-4cd8-8f20-17f0d11a8864 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:40.052712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68187a91-93fd-4cd8-8f20-17f0d11a8864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:40.052712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68187a91-93fd-4cd8-8f20-17f0d11a8864 admin admin] PATH_INFO: `/v3/projects/b65016bbd5b5491987926accbbbe8025/users/207eb77d28fa429b91e182e223c26adc/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:40.052934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb722325-fd56-4f77-8f87-114422f39eea admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:40.056425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68187a91-93fd-4cd8-8f20-17f0d11a8864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b65016bbd5b5491987926accbbbe8025, user_id=207eb77d28fa429b91e182e223c26adc, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:40.063661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68187a91-93fd-4cd8-8f20-17f0d11a8864 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:40.068770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad5f80-e26f-47c2-8fbb-2f8639938256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:40.070710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad5f80-e26f-47c2-8fbb-2f8639938256 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:40.071684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad5f80-e26f-47c2-8fbb-2f8639938256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:40.072154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad5f80-e26f-47c2-8fbb-2f8639938256 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:40.074179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad5f80-e26f-47c2-8fbb-2f8639938256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:40.086313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 939/1895] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:39 2022] PUT /identity/v3/projects/b65016bbd5b5491987926accbbbe8025/users/207eb77d28fa429b91e182e223c26adc/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:40.086562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad5f80-e26f-47c2-8fbb-2f8639938256 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:40.091612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10b32-07a0-405c-a92f-c211af707d7c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:40.094134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10b32-07a0-405c-a92f-c211af707d7c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:40.094134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f10b32-07a0-405c-a92f-c211af707d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:40.127734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-32f10b32-07a0-405c-a92f-c211af707d7c None None] MFA Rules not processed for user `207eb77d28fa429b91e182e223c26adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:40.130484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-32f10b32-07a0-405c-a92f-c211af707d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:40.207567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 957/1896] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:39 2022] POST /identity/v3/users => generated 291 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:40.212299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad5f80-e26f-47c2-8fbb-2f8639938256 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:40.226434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 940/1897] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:40.238040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad5f80-e26f-47c2-8fbb-2f8639938256 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:40.242070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10b32-07a0-405c-a92f-c211af707d7c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:40.243628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-32f10b32-07a0-405c-a92f-c211af707d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:40.320999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:40.323811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f10b32-07a0-405c-a92f-c211af707d7c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:40.328096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29e107-c5a0-4d20-b1a8-45b2153416d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:40.328983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29e107-c5a0-4d20-b1a8-45b2153416d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:40.329391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29e107-c5a0-4d20-b1a8-45b2153416d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:40.329662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def78d8-8144-4cf1-a8ed-725f8e1c71c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:40.330116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29e107-c5a0-4d20-b1a8-45b2153416d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:40.330335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def78d8-8144-4cf1-a8ed-725f8e1c71c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:40.330709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def78d8-8144-4cf1-a8ed-725f8e1c71c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:40.330997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def78d8-8144-4cf1-a8ed-725f8e1c71c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:40.331120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29e107-c5a0-4d20-b1a8-45b2153416d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:40.332860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29e107-c5a0-4d20-b1a8-45b2153416d2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:40.344195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def78d8-8144-4cf1-a8ed-725f8e1c71c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:40.350558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def78d8-8144-4cf1-a8ed-725f8e1c71c6 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:40.350745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 958/1898] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:40.352615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29e107-c5a0-4d20-b1a8-45b2153416d2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:40.362864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 941/1899] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:25:40.367744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29e107-c5a0-4d20-b1a8-45b2153416d2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:40.375935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77564e-7f70-48fe-b175-075a695dac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:40.376759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77564e-7f70-48fe-b175-075a695dac27 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:40.377150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77564e-7f70-48fe-b175-075a695dac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:40.377552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77564e-7f70-48fe-b175-075a695dac27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:40.378661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77564e-7f70-48fe-b175-075a695dac27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:40.382852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77564e-7f70-48fe-b175-075a695dac27 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:40.404984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 959/1900] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:40.409166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def78d8-8144-4cf1-a8ed-725f8e1c71c6 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:40.426178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def78d8-8144-4cf1-a8ed-725f8e1c71c6 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:40.440390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67df55a-ec5f-405d-8779-af4a4d72701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:40.441446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67df55a-ec5f-405d-8779-af4a4d72701f admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:40.441655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67df55a-ec5f-405d-8779-af4a4d72701f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:40.441971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67df55a-ec5f-405d-8779-af4a4d72701f admin admin] PATH_INFO: `/v3/projects/129e977fd9b84330823306d2d28b9a1e/users/a33272a2406b4db1bbd044f353a28034/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:40.468911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67df55a-ec5f-405d-8779-af4a4d72701f admin admin] RBAC: Authorizing `identity:create_grant(project_id=129e977fd9b84330823306d2d28b9a1e, user_id=a33272a2406b4db1bbd044f353a28034, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:40.472016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67df55a-ec5f-405d-8779-af4a4d72701f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:40.605892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 942/1901] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:40 2022] PUT /identity/v3/projects/129e977fd9b84330823306d2d28b9a1e/users/a33272a2406b4db1bbd044f353a28034/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:40.619361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e8a3f-fa16-4bcb-a5b5-f898cf520ac6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:40.620296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e8a3f-fa16-4bcb-a5b5-f898cf520ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:40.620597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e8a3f-fa16-4bcb-a5b5-f898cf520ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:40.680400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-eb9e8a3f-fa16-4bcb-a5b5-f898cf520ac6 None None] MFA Rules not processed for user `a33272a2406b4db1bbd044f353a28034`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:40.687444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-eb9e8a3f-fa16-4bcb-a5b5-f898cf520ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:40.814201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 960/1902] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:40.834140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67df55a-ec5f-405d-8779-af4a4d72701f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:40.843247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e67df55a-ec5f-405d-8779-af4a4d72701f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:40.947865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67df55a-ec5f-405d-8779-af4a4d72701f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:40.956804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b91049-dd29-4a6a-a1bd-f45394ece753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:40.957557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b91049-dd29-4a6a-a1bd-f45394ece753 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:40.957951 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b91049-dd29-4a6a-a1bd-f45394ece753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:40.958394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b91049-dd29-4a6a-a1bd-f45394ece753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:40.972395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b91049-dd29-4a6a-a1bd-f45394ece753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:40.974422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b91049-dd29-4a6a-a1bd-f45394ece753 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:40.987341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 943/1903] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:41.002833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:41.016478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e8a3f-fa16-4bcb-a5b5-f898cf520ac6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:41.221967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e8a3f-fa16-4bcb-a5b5-f898cf520ac6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:41.231787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:41.236315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:41.236899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:41.237410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] PATH_INFO: `/v3/users/a33272a2406b4db1bbd044f353a28034` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:41.239837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a33272a2406b4db1bbd044f353a28034)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:41.243737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:41.560087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a33272a2406b4db1bbd044f353a28034'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:41.569549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a33272a2406b4db1bbd044f353a28034'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:41.570479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a33272a2406b4db1bbd044f353a28034'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:41.571274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a33272a2406b4db1bbd044f353a28034'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:41.691684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a33272a2406b4db1bbd044f353a28034'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:41.731824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 961/1904] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:41 2022] DELETE /identity/v3/users/a33272a2406b4db1bbd044f353a28034 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:41.734426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ad17f-635c-45ed-a5ef-7820bb16d14a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:41.735552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ad17f-635c-45ed-a5ef-7820bb16d14a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:41.735552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ad17f-635c-45ed-a5ef-7820bb16d14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:41.761331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a62ad17f-635c-45ed-a5ef-7820bb16d14a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:41.762616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a62ad17f-635c-45ed-a5ef-7820bb16d14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:41.850453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 944/1905] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:41.873876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:41.882732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:41.989736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1b29a-cf25-48c1-9ae4-1ca54852e0d1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:41.997390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce8635-a153-4322-93d3-d06b60be8cdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:41.997963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce8635-a153-4322-93d3-d06b60be8cdc service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:41.998252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce8635-a153-4322-93d3-d06b60be8cdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:41.998567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce8635-a153-4322-93d3-d06b60be8cdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:42.044328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce8635-a153-4322-93d3-d06b60be8cdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:42.047254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce8635-a153-4322-93d3-d06b60be8cdc service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:42.061415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 962/1906] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:42.776717 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:43.010990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:43.114389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ad17f-635c-45ed-a5ef-7820bb16d14a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:43.115865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a62ad17f-635c-45ed-a5ef-7820bb16d14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:43.128597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ad17f-635c-45ed-a5ef-7820bb16d14a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:43.133857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:43.134865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:43.135099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:43.135360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] PATH_INFO: `/v3/projects/129e977fd9b84330823306d2d28b9a1e` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:43.136811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129e977fd9b84330823306d2d28b9a1e)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:43.138794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:43.292460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129e977fd9b84330823306d2d28b9a1e', 'request_id': 'req-c2c077b2-9128-4595-9e04-ce02854d04c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:43.299853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129e977fd9b84330823306d2d28b9a1e', 'request_id': 'req-c2c077b2-9128-4595-9e04-ce02854d04c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:43.381813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129e977fd9b84330823306d2d28b9a1e', 'request_id': 'req-c2c077b2-9128-4595-9e04-ce02854d04c8'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:43.398949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] The token cache is being invalidate because project 129e977fd9b84330823306d2d28b9a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:43.399376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:43.402556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c2c077b2-9128-4595-9e04-ce02854d04c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:43.404691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 945/1907] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:43 2022] DELETE /identity/v3/projects/129e977fd9b84330823306d2d28b9a1e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:43.410055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef6c39-e047-4603-a3c8-72df5ead8645 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:43.410382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef6c39-e047-4603-a3c8-72df5ead8645 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:43.410692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef6c39-e047-4603-a3c8-72df5ead8645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:43.446342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-dfef6c39-e047-4603-a3c8-72df5ead8645 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:43.451034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dfef6c39-e047-4603-a3c8-72df5ead8645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:43.547523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 963/1908] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:43.558692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f56b09-94d9-4976-b65c-4ea18d552d13 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:43.559374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f56b09-94d9-4976-b65c-4ea18d552d13 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:43.559856 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f56b09-94d9-4976-b65c-4ea18d552d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:43.580735 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-22f56b09-94d9-4976-b65c-4ea18d552d13 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:43.581984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-22f56b09-94d9-4976-b65c-4ea18d552d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:43.590944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 946/1909] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:43.596127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef6c39-e047-4603-a3c8-72df5ead8645 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:43.607989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef6c39-e047-4603-a3c8-72df5ead8645 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:43.614175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d75a11-55cb-4e37-9c26-ccbd1122f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:43.616313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d75a11-55cb-4e37-9c26-ccbd1122f8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:43.616313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d75a11-55cb-4e37-9c26-ccbd1122f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:43.616313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d75a11-55cb-4e37-9c26-ccbd1122f8d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:43.616748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d75a11-55cb-4e37-9c26-ccbd1122f8d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:43.621753 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d75a11-55cb-4e37-9c26-ccbd1122f8d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:43.621753 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d75a11-55cb-4e37-9c26-ccbd1122f8d3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:43.635280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 964/1910] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:43.639459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f56b09-94d9-4976-b65c-4ea18d552d13 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:43.660587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f56b09-94d9-4976-b65c-4ea18d552d13 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:43.666323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bb033-b16f-4bbd-84aa-56b7759b9bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:43.667194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bb033-b16f-4bbd-84aa-56b7759b9bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:43.667308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bb033-b16f-4bbd-84aa-56b7759b9bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:43.667735 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bb033-b16f-4bbd-84aa-56b7759b9bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:43.668627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bb033-b16f-4bbd-84aa-56b7759b9bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:43.670795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0bb033-b16f-4bbd-84aa-56b7759b9bb5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:43.800372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 947/1911] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:43 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:43.806611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d75a11-55cb-4e37-9c26-ccbd1122f8d3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:43.824891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d75a11-55cb-4e37-9c26-ccbd1122f8d3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:43.940173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ee4df-9939-494b-963e-2b9cbd0cd316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:43.940962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ee4df-9939-494b-963e-2b9cbd0cd316 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:43.941703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ee4df-9939-494b-963e-2b9cbd0cd316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:43.941844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ee4df-9939-494b-963e-2b9cbd0cd316 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:43.943026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ee4df-9939-494b-963e-2b9cbd0cd316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:43.946014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ee4df-9939-494b-963e-2b9cbd0cd316 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:44.153756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 965/1912] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:43 2022] POST /identity/v3/users => generated 288 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:44.157014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bb033-b16f-4bbd-84aa-56b7759b9bb5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:44.175553 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bb033-b16f-4bbd-84aa-56b7759b9bb5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:44.179572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2593f-3035-44a2-8806-209e75dec3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:44.180240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2593f-3035-44a2-8806-209e75dec3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:44.180779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2593f-3035-44a2-8806-209e75dec3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:44.180779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2593f-3035-44a2-8806-209e75dec3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:44.181546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2593f-3035-44a2-8806-209e75dec3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:44.183559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2593f-3035-44a2-8806-209e75dec3f5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:44.201143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 948/1913] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:44.206759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ee4df-9939-494b-963e-2b9cbd0cd316 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:44.219833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ee4df-9939-494b-963e-2b9cbd0cd316 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:44.226936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fff22d-a43a-4d8f-b227-db66a36514fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:44.227629 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fff22d-a43a-4d8f-b227-db66a36514fc admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:44.227909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fff22d-a43a-4d8f-b227-db66a36514fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:44.228183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fff22d-a43a-4d8f-b227-db66a36514fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:44.228918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fff22d-a43a-4d8f-b227-db66a36514fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:44.230572 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fff22d-a43a-4d8f-b227-db66a36514fc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:44.245222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 966/1914] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:44.250576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2593f-3035-44a2-8806-209e75dec3f5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:44.261758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2593f-3035-44a2-8806-209e75dec3f5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:44.269949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397b893-6a30-4977-b59f-6dde2fc2d6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:44.271187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397b893-6a30-4977-b59f-6dde2fc2d6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:44.271634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397b893-6a30-4977-b59f-6dde2fc2d6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:44.272082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397b893-6a30-4977-b59f-6dde2fc2d6ef admin admin] PATH_INFO: `/v3/projects/1b5eb3b58007493b82f19ac1a1f0582a/users/8f94f9c140bc44aa953ac10834f097e3/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:44.298871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397b893-6a30-4977-b59f-6dde2fc2d6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b5eb3b58007493b82f19ac1a1f0582a, user_id=8f94f9c140bc44aa953ac10834f097e3, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:44.307726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397b893-6a30-4977-b59f-6dde2fc2d6ef admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:44.486833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 949/1915] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:44 2022] PUT /identity/v3/projects/1b5eb3b58007493b82f19ac1a1f0582a/users/8f94f9c140bc44aa953ac10834f097e3/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:44.499987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fc8d8-833c-4863-bb79-4b1ebac28066 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:44.500332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fc8d8-833c-4863-bb79-4b1ebac28066 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:44.500876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467fc8d8-833c-4863-bb79-4b1ebac28066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:44.534985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-467fc8d8-833c-4863-bb79-4b1ebac28066 None None] MFA Rules not processed for user `8f94f9c140bc44aa953ac10834f097e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:44.536260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-467fc8d8-833c-4863-bb79-4b1ebac28066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:44.605816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 967/1916] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:44.619138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397b893-6a30-4977-b59f-6dde2fc2d6ef admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:44.621606 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d397b893-6a30-4977-b59f-6dde2fc2d6ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:44.688186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397b893-6a30-4977-b59f-6dde2fc2d6ef admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:44.692833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d7fe5-fd0f-4da7-b0af-cb7ad050d531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:44.693505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d7fe5-fd0f-4da7-b0af-cb7ad050d531 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:44.693791 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d7fe5-fd0f-4da7-b0af-cb7ad050d531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:44.694086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d7fe5-fd0f-4da7-b0af-cb7ad050d531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:44.703906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d7fe5-fd0f-4da7-b0af-cb7ad050d531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:44.705374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d7fe5-fd0f-4da7-b0af-cb7ad050d531 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:44.715347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 950/1917] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:44.832796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467fc8d8-833c-4863-bb79-4b1ebac28066 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:44.836234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-467fc8d8-833c-4863-bb79-4b1ebac28066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:44.948095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467fc8d8-833c-4863-bb79-4b1ebac28066 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:44.954371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:44.955390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:44.955672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:44.955951 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] PATH_INFO: `/v3/users/099ac957cbec4602ab71504aedb62d6d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:44.957315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099ac957cbec4602ab71504aedb62d6d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:44.959377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:45.259367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099ac957cbec4602ab71504aedb62d6d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:45.268717 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099ac957cbec4602ab71504aedb62d6d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:45.269486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099ac957cbec4602ab71504aedb62d6d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:45.270246 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099ac957cbec4602ab71504aedb62d6d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:45.347518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099ac957cbec4602ab71504aedb62d6d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:45.380009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 968/1918] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:44 2022] DELETE /identity/v3/users/099ac957cbec4602ab71504aedb62d6d => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:45.384978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313530d-32b1-4611-aa72-c73a65934e8f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:45.385490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313530d-32b1-4611-aa72-c73a65934e8f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:45.385779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313530d-32b1-4611-aa72-c73a65934e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:45.410920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d313530d-32b1-4611-aa72-c73a65934e8f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:45.412396 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d313530d-32b1-4611-aa72-c73a65934e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:45.497273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 951/1919] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:45.507960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:45.510033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:45.596599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18711181-f953-486b-8d50-3aaba7f88a58 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:45.611226 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dc8e8-4f9e-4754-8c3e-3111b9e4258e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:45.611918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dc8e8-4f9e-4754-8c3e-3111b9e4258e service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:45.612239 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dc8e8-4f9e-4754-8c3e-3111b9e4258e service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:45.612522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dc8e8-4f9e-4754-8c3e-3111b9e4258e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:45.626363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237dc8e8-4f9e-4754-8c3e-3111b9e4258e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:45.627874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237dc8e8-4f9e-4754-8c3e-3111b9e4258e service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:45.663396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 969/1920] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:45.811382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:46.304032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313530d-32b1-4611-aa72-c73a65934e8f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:46.305702 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d313530d-32b1-4611-aa72-c73a65934e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:46.320131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313530d-32b1-4611-aa72-c73a65934e8f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:46.326671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:46.329222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:46.329732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:46.330221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] PATH_INFO: `/v3/projects/6f769e516f5041798aada26d25ac54c5` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:46.332587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f769e516f5041798aada26d25ac54c5)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:46.336425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:46.388209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:46.469231 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f769e516f5041798aada26d25ac54c5', 'request_id': 'req-0151ce22-d85e-4e88-b3f3-0379ef42314c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:46.474849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f769e516f5041798aada26d25ac54c5', 'request_id': 'req-0151ce22-d85e-4e88-b3f3-0379ef42314c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:46.591581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f769e516f5041798aada26d25ac54c5', 'request_id': 'req-0151ce22-d85e-4e88-b3f3-0379ef42314c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:46.605797 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] The token cache is being invalidate because project 6f769e516f5041798aada26d25ac54c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:46.606091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:46.606843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:46.608509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 952/1921] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:46 2022] DELETE /identity/v3/projects/6f769e516f5041798aada26d25ac54c5 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:46.613465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dc8e8-4f9e-4754-8c3e-3111b9e4258e service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:46.626243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dc8e8-4f9e-4754-8c3e-3111b9e4258e service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:46.694059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:46.694942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:46.695421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:46.695728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] PATH_INFO: `/v3/users/207eb77d28fa429b91e182e223c26adc` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:46.697240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=207eb77d28fa429b91e182e223c26adc)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:46.699284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:46.934236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '207eb77d28fa429b91e182e223c26adc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:46.942145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '207eb77d28fa429b91e182e223c26adc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:46.942896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '207eb77d28fa429b91e182e223c26adc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:46.943497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '207eb77d28fa429b91e182e223c26adc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:47.053960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2eab492-8ab0-491d-9627-356493707056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '207eb77d28fa429b91e182e223c26adc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:47.075308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 970/1922] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:46 2022] DELETE /identity/v3/users/207eb77d28fa429b91e182e223c26adc => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:47.664124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:47.665789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:47.680327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151ce22-d85e-4e88-b3f3-0379ef42314c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:47.748110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:47.749045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:47.749333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:47.749619 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] PATH_INFO: `/v3/projects/b65016bbd5b5491987926accbbbe8025` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:47.751033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b65016bbd5b5491987926accbbbe8025)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:47.753005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:47.965494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b65016bbd5b5491987926accbbbe8025', 'request_id': 'req-29043a76-aba7-4294-b0d7-a7fed73373c5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:47.971643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b65016bbd5b5491987926accbbbe8025', 'request_id': 'req-29043a76-aba7-4294-b0d7-a7fed73373c5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:48.080002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b65016bbd5b5491987926accbbbe8025', 'request_id': 'req-29043a76-aba7-4294-b0d7-a7fed73373c5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:48.098095 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] The token cache is being invalidate because project b65016bbd5b5491987926accbbbe8025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:48.101343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:48.103983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-29043a76-aba7-4294-b0d7-a7fed73373c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:48.106124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 953/1923] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:47 2022] DELETE /identity/v3/projects/b65016bbd5b5491987926accbbbe8025 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:48.112993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00943815-53ce-48e0-b78c-b207471ce718 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:48.113971 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00943815-53ce-48e0-b78c-b207471ce718 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:48.114480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00943815-53ce-48e0-b78c-b207471ce718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:48.136252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-00943815-53ce-48e0-b78c-b207471ce718 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:48.139793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-00943815-53ce-48e0-b78c-b207471ce718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:48.225781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 971/1924] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:48.231080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2ce72-587b-44c9-8b88-6296ad140c23 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:48.231624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2ce72-587b-44c9-8b88-6296ad140c23 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:48.231957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2ce72-587b-44c9-8b88-6296ad140c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:48.274313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6ea2ce72-587b-44c9-8b88-6296ad140c23 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:48.276394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6ea2ce72-587b-44c9-8b88-6296ad140c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:48.295841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00943815-53ce-48e0-b78c-b207471ce718 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:48.312709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 954/1925] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:48.312807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00943815-53ce-48e0-b78c-b207471ce718 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:48.318393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf56280-9322-4b0d-bb2a-94e955c4f5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:48.319445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf56280-9322-4b0d-bb2a-94e955c4f5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:48.320503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf56280-9322-4b0d-bb2a-94e955c4f5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:48.320968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf56280-9322-4b0d-bb2a-94e955c4f5f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:48.321931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf56280-9322-4b0d-bb2a-94e955c4f5f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:48.322527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf56280-9322-4b0d-bb2a-94e955c4f5f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:48.324428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf56280-9322-4b0d-bb2a-94e955c4f5f8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:48.343551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 972/1926] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:48.347838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2ce72-587b-44c9-8b88-6296ad140c23 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:48.371314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2ce72-587b-44c9-8b88-6296ad140c23 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:48.382516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91b941-6890-4af1-a875-4e407269c91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:48.383852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91b941-6890-4af1-a875-4e407269c91e admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:48.384140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91b941-6890-4af1-a875-4e407269c91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:48.384561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91b941-6890-4af1-a875-4e407269c91e admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:48.385184 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91b941-6890-4af1-a875-4e407269c91e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:48.387544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91b941-6890-4af1-a875-4e407269c91e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:48.832316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 955/1927] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:48 2022] POST /identity/v3/projects => generated 387 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:48.836344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf56280-9322-4b0d-bb2a-94e955c4f5f8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:48.853468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf56280-9322-4b0d-bb2a-94e955c4f5f8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:48.938577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07c708-9939-439f-ac23-de41b412f27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:48.939626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07c708-9939-439f-ac23-de41b412f27b admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:48.939626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07c708-9939-439f-ac23-de41b412f27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:48.939900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07c708-9939-439f-ac23-de41b412f27b admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:48.940611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07c708-9939-439f-ac23-de41b412f27b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:48.942653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07c708-9939-439f-ac23-de41b412f27b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:49.054722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 973/1928] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:48 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:49.057934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91b941-6890-4af1-a875-4e407269c91e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:49.071658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91b941-6890-4af1-a875-4e407269c91e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:49.076364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a4dc0-82b4-410d-b466-1d93cc887f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:49.077048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a4dc0-82b4-410d-b466-1d93cc887f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:49.077334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a4dc0-82b4-410d-b466-1d93cc887f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:49.077612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a4dc0-82b4-410d-b466-1d93cc887f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:49.078529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a4dc0-82b4-410d-b466-1d93cc887f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:49.080179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a4dc0-82b4-410d-b466-1d93cc887f5c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:49.092130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 956/1929] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:49.097474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07c708-9939-439f-ac23-de41b412f27b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:49.109857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07c708-9939-439f-ac23-de41b412f27b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:49.115176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d0810-ab7c-4978-821d-da94db020278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:49.115863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d0810-ab7c-4978-821d-da94db020278 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:49.115863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d0810-ab7c-4978-821d-da94db020278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:49.116436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d0810-ab7c-4978-821d-da94db020278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:49.117135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d0810-ab7c-4978-821d-da94db020278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:49.118882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d0810-ab7c-4978-821d-da94db020278 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:49.130200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 974/1930] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:49.134252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a4dc0-82b4-410d-b466-1d93cc887f5c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:49.146273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a4dc0-82b4-410d-b466-1d93cc887f5c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:49.152036 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d6b17-19ac-40fa-a756-777297e2b51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:49.152036 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6b17-19ac-40fa-a756-777297e2b51d admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:49.152353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6b17-19ac-40fa-a756-777297e2b51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:49.152638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d6b17-19ac-40fa-a756-777297e2b51d admin admin] PATH_INFO: `/v3/projects/0eeb5a4dfefd4c48933acb0784553dd2/users/375cd6d0b30c44d7aaa5c8e65d985ee4/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:49.169511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d6b17-19ac-40fa-a756-777297e2b51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eeb5a4dfefd4c48933acb0784553dd2, user_id=375cd6d0b30c44d7aaa5c8e65d985ee4, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:49.173085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d6b17-19ac-40fa-a756-777297e2b51d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:49.328993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 957/1931] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:49 2022] PUT /identity/v3/projects/0eeb5a4dfefd4c48933acb0784553dd2/users/375cd6d0b30c44d7aaa5c8e65d985ee4/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:49.343666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ad2dc-e2af-4e51-99e5-e8d67c44a761 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:49.343985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ad2dc-e2af-4e51-99e5-e8d67c44a761 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:49.344297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ad2dc-e2af-4e51-99e5-e8d67c44a761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:49.372892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-526ad2dc-e2af-4e51-99e5-e8d67c44a761 None None] MFA Rules not processed for user `375cd6d0b30c44d7aaa5c8e65d985ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:49.375148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-526ad2dc-e2af-4e51-99e5-e8d67c44a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:49.474792 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 975/1932] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:49.479651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d6b17-19ac-40fa-a756-777297e2b51d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:49.560298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d6b17-19ac-40fa-a756-777297e2b51d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:49.564935 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c86ac-2c7d-4c08-a34f-25ec7a62aed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:49.565631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c86ac-2c7d-4c08-a34f-25ec7a62aed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:49.566054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c86ac-2c7d-4c08-a34f-25ec7a62aed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:49.566198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c86ac-2c7d-4c08-a34f-25ec7a62aed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:49.567084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c86ac-2c7d-4c08-a34f-25ec7a62aed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:49.569195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c86ac-2c7d-4c08-a34f-25ec7a62aed4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:49.722958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 958/1933] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:49 2022] POST /identity/v3/projects => generated 385 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:49.727234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ad2dc-e2af-4e51-99e5-e8d67c44a761 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:49.738900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ad2dc-e2af-4e51-99e5-e8d67c44a761 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:49.821446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf1193-7c92-44a4-84cf-e72b078c8232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:49.821446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1193-7c92-44a4-84cf-e72b078c8232 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:49.821446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1193-7c92-44a4-84cf-e72b078c8232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:49.821446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1193-7c92-44a4-84cf-e72b078c8232 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:49.821843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf1193-7c92-44a4-84cf-e72b078c8232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:49.823260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf1193-7c92-44a4-84cf-e72b078c8232 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:49.935545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 976/1934] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:49 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:49.939725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c86ac-2c7d-4c08-a34f-25ec7a62aed4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:49.954700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c86ac-2c7d-4c08-a34f-25ec7a62aed4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:49.962198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa94c7-5278-4c84-af88-78fef921e072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:49.963085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa94c7-5278-4c84-af88-78fef921e072 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:49.963533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa94c7-5278-4c84-af88-78fef921e072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:49.963840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa94c7-5278-4c84-af88-78fef921e072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:49.965034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa94c7-5278-4c84-af88-78fef921e072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:49.977196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa94c7-5278-4c84-af88-78fef921e072 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:49.992016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 959/1935] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:49.996211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf1193-7c92-44a4-84cf-e72b078c8232 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:50.008057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf1193-7c92-44a4-84cf-e72b078c8232 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:50.012906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263a68c-8d83-4c9b-982f-4a1367014fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:50.013815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263a68c-8d83-4c9b-982f-4a1367014fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:50.014301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263a68c-8d83-4c9b-982f-4a1367014fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:50.014843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263a68c-8d83-4c9b-982f-4a1367014fc9 admin admin] PATH_INFO: `/v3/projects/2003cf15930a49028524e5a9efb40baf/users/6bb2e76db8d342c9a57497af664659d7/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:50.033235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263a68c-8d83-4c9b-982f-4a1367014fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2003cf15930a49028524e5a9efb40baf, user_id=6bb2e76db8d342c9a57497af664659d7, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:50.038727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263a68c-8d83-4c9b-982f-4a1367014fc9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:50.161290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 977/1936] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:49 2022] PUT /identity/v3/projects/2003cf15930a49028524e5a9efb40baf/users/6bb2e76db8d342c9a57497af664659d7/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:50.169043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa94c7-5278-4c84-af88-78fef921e072 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:50.183664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa94c7-5278-4c84-af88-78fef921e072 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:50.251314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9075d8-9039-4e7d-a54a-73fccf807fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:50.252177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9075d8-9039-4e7d-a54a-73fccf807fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:50.252578 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9075d8-9039-4e7d-a54a-73fccf807fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:50.252960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9075d8-9039-4e7d-a54a-73fccf807fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:50.253832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9075d8-9039-4e7d-a54a-73fccf807fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:50.256301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9075d8-9039-4e7d-a54a-73fccf807fcf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:50.269103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 960/1937] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:50.275043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263a68c-8d83-4c9b-982f-4a1367014fc9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:50.290532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263a68c-8d83-4c9b-982f-4a1367014fc9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:50.296356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2677fa-6608-4cae-9cb8-a5118ac57a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:50.297369 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2677fa-6608-4cae-9cb8-a5118ac57a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:50.297658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2677fa-6608-4cae-9cb8-a5118ac57a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:50.298025 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2677fa-6608-4cae-9cb8-a5118ac57a99 admin admin] PATH_INFO: `/v3/projects/2003cf15930a49028524e5a9efb40baf/users/6bb2e76db8d342c9a57497af664659d7/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:50.300612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2677fa-6608-4cae-9cb8-a5118ac57a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2003cf15930a49028524e5a9efb40baf, user_id=6bb2e76db8d342c9a57497af664659d7, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:50.303414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2677fa-6608-4cae-9cb8-a5118ac57a99 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:50.316305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 978/1938] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:50 2022] PUT /identity/v3/projects/2003cf15930a49028524e5a9efb40baf/users/6bb2e76db8d342c9a57497af664659d7/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:50.328185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f4b6a-7434-450a-9b76-69ec4f4141dc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:50.328601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f4b6a-7434-450a-9b76-69ec4f4141dc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:50.328927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f4b6a-7434-450a-9b76-69ec4f4141dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:50.361724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b20f4b6a-7434-450a-9b76-69ec4f4141dc None None] MFA Rules not processed for user `6bb2e76db8d342c9a57497af664659d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:50.363020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b20f4b6a-7434-450a-9b76-69ec4f4141dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:50.446536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 961/1939] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:50.464458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2677fa-6608-4cae-9cb8-a5118ac57a99 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:50.466676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fe2677fa-6608-4cae-9cb8-a5118ac57a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:50.558931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2677fa-6608-4cae-9cb8-a5118ac57a99 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:50.566373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3313803-4624-40bd-9365-4a44ad9f3a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:50.567484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3313803-4624-40bd-9365-4a44ad9f3a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:50.567966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3313803-4624-40bd-9365-4a44ad9f3a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:50.568442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3313803-4624-40bd-9365-4a44ad9f3a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:50.588778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3313803-4624-40bd-9365-4a44ad9f3a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:50.591737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3313803-4624-40bd-9365-4a44ad9f3a2b service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:50.607603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 979/1940] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:50.644624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f4b6a-7434-450a-9b76-69ec4f4141dc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:50.774333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f4b6a-7434-450a-9b76-69ec4f4141dc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:50.781687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:50.782789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:50.783180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:50.783758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] PATH_INFO: `/v3/users/375cd6d0b30c44d7aaa5c8e65d985ee4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:50.785552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=375cd6d0b30c44d7aaa5c8e65d985ee4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:50.788502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:50.871041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:51.148712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3313803-4624-40bd-9365-4a44ad9f3a2b service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:51.153172 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d3313803-4624-40bd-9365-4a44ad9f3a2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:51.175819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3313803-4624-40bd-9365-4a44ad9f3a2b service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:51.186118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:51.186118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:51.186118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:51.186118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] PATH_INFO: `/v3/users/8f94f9c140bc44aa953ac10834f097e3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:51.186835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f94f9c140bc44aa953ac10834f097e3)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:51.186835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:51.188839 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '375cd6d0b30c44d7aaa5c8e65d985ee4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.189646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '375cd6d0b30c44d7aaa5c8e65d985ee4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.216670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '375cd6d0b30c44d7aaa5c8e65d985ee4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.218623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '375cd6d0b30c44d7aaa5c8e65d985ee4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.417249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:51.418321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-abb6d084-44ca-475c-8a7c-7a7f28d72592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '375cd6d0b30c44d7aaa5c8e65d985ee4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.433721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 962/1941] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:50 2022] DELETE /identity/v3/users/375cd6d0b30c44d7aaa5c8e65d985ee4 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:51.438807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ff6a-5fca-4d68-bdf0-bdc5f7cc0776 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:51.439023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ff6a-5fca-4d68-bdf0-bdc5f7cc0776 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:51.439310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ff6a-5fca-4d68-bdf0-bdc5f7cc0776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:51.457597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3d57ff6a-5fca-4d68-bdf0-bdc5f7cc0776 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:51.458867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3d57ff6a-5fca-4d68-bdf0-bdc5f7cc0776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:51.524288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 963/1942] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:51.535054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ff6a-5fca-4d68-bdf0-bdc5f7cc0776 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:51.536544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3d57ff6a-5fca-4d68-bdf0-bdc5f7cc0776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:51.599682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ff6a-5fca-4d68-bdf0-bdc5f7cc0776 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:51.603400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be568235-cc61-4b8e-93cc-0174a6c6dc3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:51.604057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be568235-cc61-4b8e-93cc-0174a6c6dc3e service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:51.604338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be568235-cc61-4b8e-93cc-0174a6c6dc3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:51.604623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be568235-cc61-4b8e-93cc-0174a6c6dc3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:51.618180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be568235-cc61-4b8e-93cc-0174a6c6dc3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:51.621297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be568235-cc61-4b8e-93cc-0174a6c6dc3e service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:51.641033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 964/1943] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:25:51.762942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f94f9c140bc44aa953ac10834f097e3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.770456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f94f9c140bc44aa953ac10834f097e3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.771177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f94f9c140bc44aa953ac10834f097e3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.771792 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f94f9c140bc44aa953ac10834f097e3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.931990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f94f9c140bc44aa953ac10834f097e3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:51.965935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 980/1944] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:51 2022] DELETE /identity/v3/users/8f94f9c140bc44aa953ac10834f097e3 => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:51.973843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e7c52-7741-47fe-9d3e-0530f7947972 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:51.974627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e7c52-7741-47fe-9d3e-0530f7947972 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:51.975319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e7c52-7741-47fe-9d3e-0530f7947972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:52.022140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b14e7c52-7741-47fe-9d3e-0530f7947972 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:52.022140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b14e7c52-7741-47fe-9d3e-0530f7947972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:52.109332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 965/1945] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:52.120199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:52.121635 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:52.192574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb77c1-81d9-4795-8f9e-2318df6cdd3c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:52.196608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e576-5f36-4e65-af55-94ae1c30d907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:52.197275 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e576-5f36-4e65-af55-94ae1c30d907 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:52.197702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e576-5f36-4e65-af55-94ae1c30d907 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:52.198012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e576-5f36-4e65-af55-94ae1c30d907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:52.208676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3e576-5f36-4e65-af55-94ae1c30d907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:52.210170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3e576-5f36-4e65-af55-94ae1c30d907 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:52.221566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 981/1946] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:52.329343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e7c52-7741-47fe-9d3e-0530f7947972 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:52.330872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b14e7c52-7741-47fe-9d3e-0530f7947972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:52.344535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e7c52-7741-47fe-9d3e-0530f7947972 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:52.349865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:52.350821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:52.351073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:52.351993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] PATH_INFO: `/v3/projects/0eeb5a4dfefd4c48933acb0784553dd2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:52.352749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eeb5a4dfefd4c48933acb0784553dd2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:52.354773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:52.588480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eeb5a4dfefd4c48933acb0784553dd2', 'request_id': 'req-107cd561-eb63-4555-b2b3-2f30d499e3f1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:52.596686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eeb5a4dfefd4c48933acb0784553dd2', 'request_id': 'req-107cd561-eb63-4555-b2b3-2f30d499e3f1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:52.741925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eeb5a4dfefd4c48933acb0784553dd2', 'request_id': 'req-107cd561-eb63-4555-b2b3-2f30d499e3f1'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:52.753992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] The token cache is being invalidate because project 0eeb5a4dfefd4c48933acb0784553dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:52.754282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:52.754998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:52.756604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 966/1947] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:52 2022] DELETE /identity/v3/projects/0eeb5a4dfefd4c48933acb0784553dd2 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:52.761099 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e576-5f36-4e65-af55-94ae1c30d907 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:52.772039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e576-5f36-4e65-af55-94ae1c30d907 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:52.833315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:52.834103 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:52.834388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:52.834792 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] PATH_INFO: `/v3/users/6bb2e76db8d342c9a57497af664659d7` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:52.835979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb2e76db8d342c9a57497af664659d7)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:52.837957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:52.857779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:52.861835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:52.882311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cd561-eb63-4555-b2b3-2f30d499e3f1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:52.890614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:52.892304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:52.892565 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:52.892901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] PATH_INFO: `/v3/projects/1b5eb3b58007493b82f19ac1a1f0582a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:52.895211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b5eb3b58007493b82f19ac1a1f0582a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:52.897154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:53.141549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb2e76db8d342c9a57497af664659d7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.141956 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b5eb3b58007493b82f19ac1a1f0582a', 'request_id': 'req-07513874-dafa-452a-87c4-2cc7d66a593a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.149564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b5eb3b58007493b82f19ac1a1f0582a', 'request_id': 'req-07513874-dafa-452a-87c4-2cc7d66a593a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.161816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb2e76db8d342c9a57497af664659d7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.163004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb2e76db8d342c9a57497af664659d7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.164328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb2e76db8d342c9a57497af664659d7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.273432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b5eb3b58007493b82f19ac1a1f0582a', 'request_id': 'req-07513874-dafa-452a-87c4-2cc7d66a593a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.299917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] The token cache is being invalidate because project 1b5eb3b58007493b82f19ac1a1f0582a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:53.300447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.301659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-07513874-dafa-452a-87c4-2cc7d66a593a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.304888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 967/1948] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:52 2022] DELETE /identity/v3/projects/1b5eb3b58007493b82f19ac1a1f0582a => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:53.315140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb8009-ea3e-4093-9702-f7483df7a305 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:53.315591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb8009-ea3e-4093-9702-f7483df7a305 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:53.316023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb8009-ea3e-4093-9702-f7483df7a305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:53.346365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b7eb8009-ea3e-4093-9702-f7483df7a305 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:53.348581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b7eb8009-ea3e-4093-9702-f7483df7a305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:53.406191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-888e6434-beea-4f8a-8a1f-7959e83b284c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb2e76db8d342c9a57497af664659d7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:53.435015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 982/1949] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:52 2022] DELETE /identity/v3/users/6bb2e76db8d342c9a57497af664659d7 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:53.487793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 968/1950] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:53.491678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a4cb1-ccc8-42e8-b092-bbe129e90321 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:53.491968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a4cb1-ccc8-42e8-b092-bbe129e90321 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:53.492856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565a4cb1-ccc8-42e8-b092-bbe129e90321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:53.516293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:53.524393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-565a4cb1-ccc8-42e8-b092-bbe129e90321 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:53.526285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-565a4cb1-ccc8-42e8-b092-bbe129e90321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:53.613996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 983/1951] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:53.618428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb8009-ea3e-4093-9702-f7483df7a305 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:53.629260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb8009-ea3e-4093-9702-f7483df7a305 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:53.634273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e067512-4996-49c7-bcc1-b43ba24012b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:53.634986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e067512-4996-49c7-bcc1-b43ba24012b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:53.635269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e067512-4996-49c7-bcc1-b43ba24012b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:53.635540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e067512-4996-49c7-bcc1-b43ba24012b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:53.636279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e067512-4996-49c7-bcc1-b43ba24012b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:53.636589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e067512-4996-49c7-bcc1-b43ba24012b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:53.638360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e067512-4996-49c7-bcc1-b43ba24012b0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:53.651676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 969/1952] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:53.658796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a4cb1-ccc8-42e8-b092-bbe129e90321 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:53.669771 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565a4cb1-ccc8-42e8-b092-bbe129e90321 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:53.675341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de58713-d0f8-4af9-b822-8deaaff7a312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:53.676125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de58713-d0f8-4af9-b822-8deaaff7a312 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:53.676421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de58713-d0f8-4af9-b822-8deaaff7a312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:53.676703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de58713-d0f8-4af9-b822-8deaaff7a312 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:53.677428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de58713-d0f8-4af9-b822-8deaaff7a312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:53.679461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de58713-d0f8-4af9-b822-8deaaff7a312 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:53.822673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 984/1953] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:53 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:53.827241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e067512-4996-49c7-bcc1-b43ba24012b0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:53.839493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e067512-4996-49c7-bcc1-b43ba24012b0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:53.922239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9331ed-d9fa-4fdd-b20f-f17286c65f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:53.922239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9331ed-d9fa-4fdd-b20f-f17286c65f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:53.922239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9331ed-d9fa-4fdd-b20f-f17286c65f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:53.922239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9331ed-d9fa-4fdd-b20f-f17286c65f20 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:53.923063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9331ed-d9fa-4fdd-b20f-f17286c65f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:53.923063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9331ed-d9fa-4fdd-b20f-f17286c65f20 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:54.206715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de58713-d0f8-4af9-b822-8deaaff7a312 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:54.208508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7de58713-d0f8-4af9-b822-8deaaff7a312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:54.226684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de58713-d0f8-4af9-b822-8deaaff7a312 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:54.231697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:54.232653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.232944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.234446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] PATH_INFO: `/v3/projects/2003cf15930a49028524e5a9efb40baf` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.235852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2003cf15930a49028524e5a9efb40baf)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:54.237917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:54.258475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 970/1954] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:53 2022] POST /identity/v3/users => generated 289 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:54.271162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9331ed-d9fa-4fdd-b20f-f17286c65f20 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:54.282321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9331ed-d9fa-4fdd-b20f-f17286c65f20 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:54.287246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142f3e1-8ac9-48e7-9879-846271021614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:54.287924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142f3e1-8ac9-48e7-9879-846271021614 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.288206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142f3e1-8ac9-48e7-9879-846271021614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.288481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142f3e1-8ac9-48e7-9879-846271021614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.289350 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142f3e1-8ac9-48e7-9879-846271021614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:54.290998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142f3e1-8ac9-48e7-9879-846271021614 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:54.302968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 971/1955] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:54.307072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142f3e1-8ac9-48e7-9879-846271021614 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:54.322759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142f3e1-8ac9-48e7-9879-846271021614 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:54.328859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926613a-97a5-46a5-bc3c-87a996be54bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:54.329537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926613a-97a5-46a5-bc3c-87a996be54bd admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.329827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926613a-97a5-46a5-bc3c-87a996be54bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.330144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926613a-97a5-46a5-bc3c-87a996be54bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.330924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926613a-97a5-46a5-bc3c-87a996be54bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:54.332872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926613a-97a5-46a5-bc3c-87a996be54bd admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:54.347640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 972/1956] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:54.351826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926613a-97a5-46a5-bc3c-87a996be54bd admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:54.362986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926613a-97a5-46a5-bc3c-87a996be54bd admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:54.367862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d17f30-a607-4618-b770-e2bb2d1f18d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:54.368768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d17f30-a607-4618-b770-e2bb2d1f18d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.369056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d17f30-a607-4618-b770-e2bb2d1f18d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.369339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d17f30-a607-4618-b770-e2bb2d1f18d6 admin admin] PATH_INFO: `/v3/projects/eec0810c2e354026a983ca1f586c33c1/users/5b22ebc6ae074c8fb1fedc593a3b6ab0/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.388997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d17f30-a607-4618-b770-e2bb2d1f18d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec0810c2e354026a983ca1f586c33c1, user_id=5b22ebc6ae074c8fb1fedc593a3b6ab0, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:54.393104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d17f30-a607-4618-b770-e2bb2d1f18d6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:54.505480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 973/1957] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:54 2022] PUT /identity/v3/projects/eec0810c2e354026a983ca1f586c33c1/users/5b22ebc6ae074c8fb1fedc593a3b6ab0/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:54.515416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2003cf15930a49028524e5a9efb40baf', 'request_id': 'req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:54.516452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b188ee-3bea-4a1f-9d58-e9aa2749b901 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.516752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b188ee-3bea-4a1f-9d58-e9aa2749b901 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.517033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b188ee-3bea-4a1f-9d58-e9aa2749b901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.524449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2003cf15930a49028524e5a9efb40baf', 'request_id': 'req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:54.556061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b4b188ee-3bea-4a1f-9d58-e9aa2749b901 None None] MFA Rules not processed for user `5b22ebc6ae074c8fb1fedc593a3b6ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:54.557319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b4b188ee-3bea-4a1f-9d58-e9aa2749b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:54.650640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2003cf15930a49028524e5a9efb40baf', 'request_id': 'req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:54.667048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 974/1958] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:54.669945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] The token cache is being invalidate because project 2003cf15930a49028524e5a9efb40baf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:54.670303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:54.672011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-a2f9be06-b2aa-4db4-9ca6-682f0db6cac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:54.674050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b188ee-3bea-4a1f-9d58-e9aa2749b901 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:54.678016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 985/1959] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:54 2022] DELETE /identity/v3/projects/2003cf15930a49028524e5a9efb40baf => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:54.680178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6cd49-974e-4a62-a3ae-f54546f097c0 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.680485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6cd49-974e-4a62-a3ae-f54546f097c0 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.680773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6cd49-974e-4a62-a3ae-f54546f097c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.703013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-39d6cd49-974e-4a62-a3ae-f54546f097c0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:54.704352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-39d6cd49-974e-4a62-a3ae-f54546f097c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:54.734923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:54.792812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 986/1960] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:54.805316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5c346-4ae5-4840-9926-ac33f64c56c6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.805715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5c346-4ae5-4840-9926-ac33f64c56c6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.806686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b188ee-3bea-4a1f-9d58-e9aa2749b901 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:54.806866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5c346-4ae5-4840-9926-ac33f64c56c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.814887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1907e1-b349-4b8c-93dd-a89a242920de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:54.815677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1907e1-b349-4b8c-93dd-a89a242920de admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.816681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1907e1-b349-4b8c-93dd-a89a242920de admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.817054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1907e1-b349-4b8c-93dd-a89a242920de admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.818088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1907e1-b349-4b8c-93dd-a89a242920de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:54.821194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1907e1-b349-4b8c-93dd-a89a242920de admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:54.840934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-48c5c346-4ae5-4840-9926-ac33f64c56c6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:54.842747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-48c5c346-4ae5-4840-9926-ac33f64c56c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:54.853872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 987/1961] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:54.857935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5c346-4ae5-4840-9926-ac33f64c56c6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:54.868093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5c346-4ae5-4840-9926-ac33f64c56c6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:54.873419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150e391-efc3-4a6c-920c-bae9c59bd3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:54.874751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150e391-efc3-4a6c-920c-bae9c59bd3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.875032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150e391-efc3-4a6c-920c-bae9c59bd3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.875381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150e391-efc3-4a6c-920c-bae9c59bd3c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.876286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150e391-efc3-4a6c-920c-bae9c59bd3c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:25:54.876609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150e391-efc3-4a6c-920c-bae9c59bd3c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:54.878856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150e391-efc3-4a6c-920c-bae9c59bd3c8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:54.892493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 988/1962] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:25:54.899423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150e391-efc3-4a6c-920c-bae9c59bd3c8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:54.909597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150e391-efc3-4a6c-920c-bae9c59bd3c8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:54.914523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab088790-1a0a-42a8-ba3e-08aa267222d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:54.915208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab088790-1a0a-42a8-ba3e-08aa267222d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:54.915477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab088790-1a0a-42a8-ba3e-08aa267222d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:54.915761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab088790-1a0a-42a8-ba3e-08aa267222d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:54.916463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab088790-1a0a-42a8-ba3e-08aa267222d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:54.918391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab088790-1a0a-42a8-ba3e-08aa267222d7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:54.944460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 975/1963] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:54 2022] POST /identity/v3/projects => generated 373 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:54.948518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1907e1-b349-4b8c-93dd-a89a242920de admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:54.950354 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3b1907e1-b349-4b8c-93dd-a89a242920de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:55.026801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1907e1-b349-4b8c-93dd-a89a242920de admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.030140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde7836-a8de-4326-8e22-418d86be1166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.031068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde7836-a8de-4326-8e22-418d86be1166 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.031354 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde7836-a8de-4326-8e22-418d86be1166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.031655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde7836-a8de-4326-8e22-418d86be1166 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.032458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde7836-a8de-4326-8e22-418d86be1166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.034816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde7836-a8de-4326-8e22-418d86be1166 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.133607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 989/1964] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:54 2022] POST /identity/v3/projects => generated 389 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:55.137739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab088790-1a0a-42a8-ba3e-08aa267222d7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:55.218161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab088790-1a0a-42a8-ba3e-08aa267222d7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.225069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f0c86-7a3c-47ef-85b0-377d7505e47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.225816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0c86-7a3c-47ef-85b0-377d7505e47a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.226378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0c86-7a3c-47ef-85b0-377d7505e47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.226378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0c86-7a3c-47ef-85b0-377d7505e47a admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.233786 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f0c86-7a3c-47ef-85b0-377d7505e47a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.233786 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f0c86-7a3c-47ef-85b0-377d7505e47a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.234085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 976/1965] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:54 2022] POST /identity/v3/users => generated 287 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:55.253540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde7836-a8de-4326-8e22-418d86be1166 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:55.269850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde7836-a8de-4326-8e22-418d86be1166 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.279000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd46e-f678-4416-a8de-60bd634be77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.280100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd46e-f678-4416-a8de-60bd634be77a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.280581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd46e-f678-4416-a8de-60bd634be77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.281024 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd46e-f678-4416-a8de-60bd634be77a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.282474 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebd46e-f678-4416-a8de-60bd634be77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.285642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebd46e-f678-4416-a8de-60bd634be77a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.302450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 977/1966] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:55.305650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd46e-f678-4416-a8de-60bd634be77a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:55.318931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd46e-f678-4416-a8de-60bd634be77a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.323778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da61b1-79b3-4d68-881a-675f5ab3fde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.324488 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da61b1-79b3-4d68-881a-675f5ab3fde1 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.324767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da61b1-79b3-4d68-881a-675f5ab3fde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.325051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da61b1-79b3-4d68-881a-675f5ab3fde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.326284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da61b1-79b3-4d68-881a-675f5ab3fde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.328946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da61b1-79b3-4d68-881a-675f5ab3fde1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.345824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 978/1967] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:55.351748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da61b1-79b3-4d68-881a-675f5ab3fde1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:55.355299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 990/1968] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:55 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:55.356456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f0c86-7a3c-47ef-85b0-377d7505e47a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:55.376191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f0c86-7a3c-47ef-85b0-377d7505e47a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.376714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da61b1-79b3-4d68-881a-675f5ab3fde1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.384056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd9e7e-185c-43a9-9760-dd55ff30208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.385464 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd9e7e-185c-43a9-9760-dd55ff30208f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.386128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd9e7e-185c-43a9-9760-dd55ff30208f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.386755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd9e7e-185c-43a9-9760-dd55ff30208f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.388714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd9e7e-185c-43a9-9760-dd55ff30208f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.391135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5cb9-5fde-454f-9dd7-1f08ed15ec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.392890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5cb9-5fde-454f-9dd7-1f08ed15ec40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.393187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5cb9-5fde-454f-9dd7-1f08ed15ec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.393480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5cb9-5fde-454f-9dd7-1f08ed15ec40 admin admin] PATH_INFO: `/v3/projects/c2ca676e92db4f8e8d3db5da953e6045/users/82dade18aa994c2b948a0c520c9db619/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.396237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd9e7e-185c-43a9-9760-dd55ff30208f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.426910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5cb9-5fde-454f-9dd7-1f08ed15ec40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ca676e92db4f8e8d3db5da953e6045, user_id=82dade18aa994c2b948a0c520c9db619, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.427026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 991/1969] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:55.428830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5cb9-5fde-454f-9dd7-1f08ed15ec40 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.432301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd9e7e-185c-43a9-9760-dd55ff30208f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:55.448805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd9e7e-185c-43a9-9760-dd55ff30208f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.455264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b191c7-55ee-44ed-9dcb-b2628a34752b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.456108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b191c7-55ee-44ed-9dcb-b2628a34752b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.456518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b191c7-55ee-44ed-9dcb-b2628a34752b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.456894 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b191c7-55ee-44ed-9dcb-b2628a34752b admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.457882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b191c7-55ee-44ed-9dcb-b2628a34752b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.459829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b191c7-55ee-44ed-9dcb-b2628a34752b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.476386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 992/1970] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:55.481458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b191c7-55ee-44ed-9dcb-b2628a34752b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:55.494942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b191c7-55ee-44ed-9dcb-b2628a34752b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.501823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa32def-2231-4e5a-8546-f746e26bc6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.502776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa32def-2231-4e5a-8546-f746e26bc6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.503059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa32def-2231-4e5a-8546-f746e26bc6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.503327 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa32def-2231-4e5a-8546-f746e26bc6f9 admin admin] PATH_INFO: `/v3/projects/860fed3f7493426d94ee9202287eb904/users/8cb272746542470f838a90d811e35bf8/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.521643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa32def-2231-4e5a-8546-f746e26bc6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860fed3f7493426d94ee9202287eb904, user_id=8cb272746542470f838a90d811e35bf8, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.525967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa32def-2231-4e5a-8546-f746e26bc6f9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.607795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 979/1971] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:55 2022] PUT /identity/v3/projects/c2ca676e92db4f8e8d3db5da953e6045/users/82dade18aa994c2b948a0c520c9db619/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:55.628222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f239ae-80af-46d7-a6fc-546cbfc9e27a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.628757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f239ae-80af-46d7-a6fc-546cbfc9e27a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.629275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f239ae-80af-46d7-a6fc-546cbfc9e27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.668170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 993/1972] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:55 2022] PUT /identity/v3/projects/860fed3f7493426d94ee9202287eb904/users/8cb272746542470f838a90d811e35bf8/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:55.680793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c289d67-d9bb-4885-9fb1-6e0d92e5c171 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.681280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c289d67-d9bb-4885-9fb1-6e0d92e5c171 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.681850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c289d67-d9bb-4885-9fb1-6e0d92e5c171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.685325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-04f239ae-80af-46d7-a6fc-546cbfc9e27a None None] MFA Rules not processed for user `82dade18aa994c2b948a0c520c9db619`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:55.687638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-04f239ae-80af-46d7-a6fc-546cbfc9e27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:55.726754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0c289d67-d9bb-4885-9fb1-6e0d92e5c171 None None] MFA Rules not processed for user `8cb272746542470f838a90d811e35bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:55.728115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0c289d67-d9bb-4885-9fb1-6e0d92e5c171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:55.785245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 980/1973] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:55.791630 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f239ae-80af-46d7-a6fc-546cbfc9e27a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:55.826590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 994/1974] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:25:55.831103 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c289d67-d9bb-4885-9fb1-6e0d92e5c171 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:55.890540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f239ae-80af-46d7-a6fc-546cbfc9e27a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.898636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe22fe-9ae5-4ec4-9aed-cde23381bee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.899351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe22fe-9ae5-4ec4-9aed-cde23381bee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.902308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe22fe-9ae5-4ec4-9aed-cde23381bee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.902308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe22fe-9ae5-4ec4-9aed-cde23381bee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.902308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe22fe-9ae5-4ec4-9aed-cde23381bee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.902943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe22fe-9ae5-4ec4-9aed-cde23381bee5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.915189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c289d67-d9bb-4885-9fb1-6e0d92e5c171 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:55.926619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062328ad-da0e-4cfe-abd7-05c07d6cae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:55.937911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062328ad-da0e-4cfe-abd7-05c07d6cae18 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:55.938357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062328ad-da0e-4cfe-abd7-05c07d6cae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:55.939028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062328ad-da0e-4cfe-abd7-05c07d6cae18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:55.939438 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062328ad-da0e-4cfe-abd7-05c07d6cae18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:55.942381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062328ad-da0e-4cfe-abd7-05c07d6cae18 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:55.973739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:56.050665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 981/1975] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:55 2022] POST /identity/v3/projects => generated 375 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:56.056352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe22fe-9ae5-4ec4-9aed-cde23381bee5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.143019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 995/1976] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:25:55 2022] POST /identity/v3/projects => generated 389 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:25:56.148468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062328ad-da0e-4cfe-abd7-05c07d6cae18 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.153789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe22fe-9ae5-4ec4-9aed-cde23381bee5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.245716 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c6440-652f-475b-bd9a-3b2f412d3d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.245956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062328ad-da0e-4cfe-abd7-05c07d6cae18 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.247213 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c6440-652f-475b-bd9a-3b2f412d3d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.247795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c6440-652f-475b-bd9a-3b2f412d3d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.248559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c6440-652f-475b-bd9a-3b2f412d3d8b admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.251357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c6440-652f-475b-bd9a-3b2f412d3d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.261416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d2369-1692-4e78-88b9-af880e75a6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.261416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d2369-1692-4e78-88b9-af880e75a6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.261416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d2369-1692-4e78-88b9-af880e75a6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.261416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d2369-1692-4e78-88b9-af880e75a6e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.261851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c6440-652f-475b-bd9a-3b2f412d3d8b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.261941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d2369-1692-4e78-88b9-af880e75a6e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.261941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d2369-1692-4e78-88b9-af880e75a6e6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.403816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 982/1978] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:56 2022] POST /identity/v3/users => generated 294 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:56.404853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 996/1978] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:25:56 2022] POST /identity/v3/users => generated 301 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:25:56.409304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c6440-652f-475b-bd9a-3b2f412d3d8b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.409512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d2369-1692-4e78-88b9-af880e75a6e6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.424248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d2369-1692-4e78-88b9-af880e75a6e6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.425990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c6440-652f-475b-bd9a-3b2f412d3d8b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.430269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d88e7-a6bb-4755-9477-9565fad01dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.431124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d88e7-a6bb-4755-9477-9565fad01dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.431602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d88e7-a6bb-4755-9477-9565fad01dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.432055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d88e7-a6bb-4755-9477-9565fad01dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.432727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc6d99-b57f-4f7e-bb37-c0d90a807805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.433117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d88e7-a6bb-4755-9477-9565fad01dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.433416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc6d99-b57f-4f7e-bb37-c0d90a807805 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.433704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc6d99-b57f-4f7e-bb37-c0d90a807805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.434035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc6d99-b57f-4f7e-bb37-c0d90a807805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.435209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc6d99-b57f-4f7e-bb37-c0d90a807805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.435559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d88e7-a6bb-4755-9477-9565fad01dbf admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.438128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc6d99-b57f-4f7e-bb37-c0d90a807805 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.448447 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 997/1979] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:56.453084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 983/1980] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:25:56.453979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d88e7-a6bb-4755-9477-9565fad01dbf admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.457375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc6d99-b57f-4f7e-bb37-c0d90a807805 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.466932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d88e7-a6bb-4755-9477-9565fad01dbf admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.467902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc6d99-b57f-4f7e-bb37-c0d90a807805 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.473183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479c74b-a937-4b26-94c0-b701bed6e9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.473583 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747953e-7a11-4814-aabe-52421919c55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.474103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479c74b-a937-4b26-94c0-b701bed6e9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.474387 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479c74b-a937-4b26-94c0-b701bed6e9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.474697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479c74b-a937-4b26-94c0-b701bed6e9c7 admin admin] PATH_INFO: `/v3/projects/86cce31d941f4c9cbc8c9769899403c0/users/c914887aa194403f8c80660662a0f1ad/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.474867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747953e-7a11-4814-aabe-52421919c55b admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.475255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747953e-7a11-4814-aabe-52421919c55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.475682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747953e-7a11-4814-aabe-52421919c55b admin admin] PATH_INFO: `/v3/projects/1e14773bfa4a4ed4901f4d1589b4a7b8/users/e68a068d1f1643e0a7c11f43b060ae65/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.494026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747953e-7a11-4814-aabe-52421919c55b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e14773bfa4a4ed4901f4d1589b4a7b8, user_id=e68a068d1f1643e0a7c11f43b060ae65, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.496403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479c74b-a937-4b26-94c0-b701bed6e9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86cce31d941f4c9cbc8c9769899403c0, user_id=c914887aa194403f8c80660662a0f1ad, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.496848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747953e-7a11-4814-aabe-52421919c55b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.503486 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479c74b-a937-4b26-94c0-b701bed6e9c7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.643858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 998/1981] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:56 2022] PUT /identity/v3/projects/1e14773bfa4a4ed4901f4d1589b4a7b8/users/e68a068d1f1643e0a7c11f43b060ae65/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:56.649589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747953e-7a11-4814-aabe-52421919c55b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.673576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:56.680118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 984/1982] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:56 2022] PUT /identity/v3/projects/86cce31d941f4c9cbc8c9769899403c0/users/c914887aa194403f8c80660662a0f1ad/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:56.685936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479c74b-a937-4b26-94c0-b701bed6e9c7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.740502 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747953e-7a11-4814-aabe-52421919c55b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.811120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479c74b-a937-4b26-94c0-b701bed6e9c7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.827841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24219c-bdba-48cb-ba83-721a79022560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.828843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24219c-bdba-48cb-ba83-721a79022560 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.829012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec15dea-89fc-4c9a-bea9-e2fdf4bf2cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.829328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24219c-bdba-48cb-ba83-721a79022560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.830172 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24219c-bdba-48cb-ba83-721a79022560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.830358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec15dea-89fc-4c9a-bea9-e2fdf4bf2cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.830640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec15dea-89fc-4c9a-bea9-e2fdf4bf2cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.830802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24219c-bdba-48cb-ba83-721a79022560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.831136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec15dea-89fc-4c9a-bea9-e2fdf4bf2cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.832685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec15dea-89fc-4c9a-bea9-e2fdf4bf2cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.832877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24219c-bdba-48cb-ba83-721a79022560 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.835997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec15dea-89fc-4c9a-bea9-e2fdf4bf2cc4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.845089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 999/1983] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:56.850813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24219c-bdba-48cb-ba83-721a79022560 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.853594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 985/1984] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:25:56.861252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec15dea-89fc-4c9a-bea9-e2fdf4bf2cc4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:56.867482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24219c-bdba-48cb-ba83-721a79022560 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.872937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe8d8d-2d72-4d0a-a04b-403d9dd2411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.876553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe8d8d-2d72-4d0a-a04b-403d9dd2411a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.876553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe8d8d-2d72-4d0a-a04b-403d9dd2411a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.876553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe8d8d-2d72-4d0a-a04b-403d9dd2411a admin admin] PATH_INFO: `/v3/projects/1e14773bfa4a4ed4901f4d1589b4a7b8/users/e68a068d1f1643e0a7c11f43b060ae65/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.877521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe8d8d-2d72-4d0a-a04b-403d9dd2411a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e14773bfa4a4ed4901f4d1589b4a7b8, user_id=e68a068d1f1643e0a7c11f43b060ae65, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.880385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe8d8d-2d72-4d0a-a04b-403d9dd2411a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.881082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec15dea-89fc-4c9a-bea9-e2fdf4bf2cc4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:56.888623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54abcee-c47b-46d9-8984-87b03385f403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:56.889569 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54abcee-c47b-46d9-8984-87b03385f403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.889865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54abcee-c47b-46d9-8984-87b03385f403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.890136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54abcee-c47b-46d9-8984-87b03385f403 admin admin] PATH_INFO: `/v3/projects/86cce31d941f4c9cbc8c9769899403c0/users/c914887aa194403f8c80660662a0f1ad/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.892322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54abcee-c47b-46d9-8984-87b03385f403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86cce31d941f4c9cbc8c9769899403c0, user_id=c914887aa194403f8c80660662a0f1ad, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:56.895584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54abcee-c47b-46d9-8984-87b03385f403 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:56.917031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1000/1985] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:56 2022] PUT /identity/v3/projects/1e14773bfa4a4ed4901f4d1589b4a7b8/users/e68a068d1f1643e0a7c11f43b060ae65/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:25:56.917323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 986/1986] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:25:56 2022] PUT /identity/v3/projects/86cce31d941f4c9cbc8c9769899403c0/users/c914887aa194403f8c80660662a0f1ad/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:56.917483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929564b-f010-44e9-a745-628bc5b0a296 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.917483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929564b-f010-44e9-a745-628bc5b0a296 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.917483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929564b-f010-44e9-a745-628bc5b0a296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.926953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65915b-e709-44d5-85ad-74a3b50cac62 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:56.927383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65915b-e709-44d5-85ad-74a3b50cac62 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:56.927763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65915b-e709-44d5-85ad-74a3b50cac62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:56.967274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5c65915b-e709-44d5-85ad-74a3b50cac62 None None] MFA Rules not processed for user `c914887aa194403f8c80660662a0f1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:56.968574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5c65915b-e709-44d5-85ad-74a3b50cac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:56.973284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a929564b-f010-44e9-a745-628bc5b0a296 None None] MFA Rules not processed for user `e68a068d1f1643e0a7c11f43b060ae65`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:56.975138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a929564b-f010-44e9-a745-628bc5b0a296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:57.082049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1001/1987] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:57.094322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929564b-f010-44e9-a745-628bc5b0a296 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:57.095994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a929564b-f010-44e9-a745-628bc5b0a296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:57.098799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 987/1988] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:57.117021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65915b-e709-44d5-85ad-74a3b50cac62 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:57.118810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5c65915b-e709-44d5-85ad-74a3b50cac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:57.191796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929564b-f010-44e9-a745-628bc5b0a296 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:57.192097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65915b-e709-44d5-85ad-74a3b50cac62 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:57.197975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00276a2-54fd-41fd-8f40-95f19e788f65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:57.198799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00276a2-54fd-41fd-8f40-95f19e788f65 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:57.199084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00276a2-54fd-41fd-8f40-95f19e788f65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:57.199362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00276a2-54fd-41fd-8f40-95f19e788f65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:57.201928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c00276a2-54fd-41fd-8f40-95f19e788f65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:57.204905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6f50f-8162-41da-9d5c-e77bc0b9778d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:57.206678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6f50f-8162-41da-9d5c-e77bc0b9778d service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:57.207176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6f50f-8162-41da-9d5c-e77bc0b9778d service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:57.208082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6f50f-8162-41da-9d5c-e77bc0b9778d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:57.230863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6f50f-8162-41da-9d5c-e77bc0b9778d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:57.235285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6f50f-8162-41da-9d5c-e77bc0b9778d service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:57.258634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1002/1989] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:57.265844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:57.291515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00276a2-54fd-41fd-8f40-95f19e788f65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:57.294864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00276a2-54fd-41fd-8f40-95f19e788f65 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:57.309663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:57.316304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 988/1990] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:57.426468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:57.456547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6f50f-8162-41da-9d5c-e77bc0b9778d service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:57.566747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6f50f-8162-41da-9d5c-e77bc0b9778d service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:57.574000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:57.575319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:57.575602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:57.575879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] PATH_INFO: `/v3/users/8cb272746542470f838a90d811e35bf8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:57.578077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb272746542470f838a90d811e35bf8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:57.579788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:57.663781 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:57.890630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb272746542470f838a90d811e35bf8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:57.899554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb272746542470f838a90d811e35bf8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:57.900553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb272746542470f838a90d811e35bf8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:57.901174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb272746542470f838a90d811e35bf8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:57.977345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb272746542470f838a90d811e35bf8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:58.000601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1003/1991] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:57 2022] DELETE /identity/v3/users/8cb272746542470f838a90d811e35bf8 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:58.005361 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f4069-ac38-4c82-98e4-d4b901adcd2c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:58.005655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f4069-ac38-4c82-98e4-d4b901adcd2c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:58.006003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f4069-ac38-4c82-98e4-d4b901adcd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:58.027275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4c9f4069-ac38-4c82-98e4-d4b901adcd2c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:25:58.028906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4c9f4069-ac38-4c82-98e4-d4b901adcd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:58.113175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 989/1992] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:25:58.124133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:58.125830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:58.189746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77726186-8b12-4c14-81a0-272ba322af36 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:58.193873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ddd81-8c71-47b3-b504-d7daed4a5f1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:58.194636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ddd81-8c71-47b3-b504-d7daed4a5f1e service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:58.194925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ddd81-8c71-47b3-b504-d7daed4a5f1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:58.195205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ddd81-8c71-47b3-b504-d7daed4a5f1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:58.210310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ddd81-8c71-47b3-b504-d7daed4a5f1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:58.212133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ddd81-8c71-47b3-b504-d7daed4a5f1e service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:58.223781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1004/1993] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:25:58.335704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:58.926228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f4069-ac38-4c82-98e4-d4b901adcd2c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:58.937592 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4c9f4069-ac38-4c82-98e4-d4b901adcd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:25:58.940704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:58.953965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f4069-ac38-4c82-98e4-d4b901adcd2c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:58.961404 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:58.962668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:58.962912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:58.963257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] PATH_INFO: `/v3/projects/860fed3f7493426d94ee9202287eb904` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:58.964995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=860fed3f7493426d94ee9202287eb904)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:58.966998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:59.167793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860fed3f7493426d94ee9202287eb904', 'request_id': 'req-23a7bbab-8772-4636-a83b-d6096f3cfb4a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.175029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860fed3f7493426d94ee9202287eb904', 'request_id': 'req-23a7bbab-8772-4636-a83b-d6096f3cfb4a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.339751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860fed3f7493426d94ee9202287eb904', 'request_id': 'req-23a7bbab-8772-4636-a83b-d6096f3cfb4a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.352596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] The token cache is being invalidate because project 860fed3f7493426d94ee9202287eb904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:25:59.352877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.353579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.355285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 990/1994] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:25:58 2022] DELETE /identity/v3/projects/860fed3f7493426d94ee9202287eb904 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:25:59.364153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ddd81-8c71-47b3-b504-d7daed4a5f1e service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:25:59.381824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ddd81-8c71-47b3-b504-d7daed4a5f1e service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:25:59.441241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:25:59.485410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:25:59.486837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:25:59.487198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:25:59.487548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] PATH_INFO: `/v3/users/e68a068d1f1643e0a7c11f43b060ae65` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:25:59.489170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68a068d1f1643e0a7c11f43b060ae65)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:25:59.491363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:25:59.751056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68a068d1f1643e0a7c11f43b060ae65'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.762475 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68a068d1f1643e0a7c11f43b060ae65'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.763450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68a068d1f1643e0a7c11f43b060ae65'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.764217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68a068d1f1643e0a7c11f43b060ae65'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.914528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68a068d1f1643e0a7c11f43b060ae65'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:25:59.942685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1005/1995] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:25:59 2022] DELETE /identity/v3/users/e68a068d1f1643e0a7c11f43b060ae65 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:00.059698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:00.530569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:00.679507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:00.682433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:00.710595 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7bbab-8772-4636-a83b-d6096f3cfb4a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:00.774868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:00.855601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:00.857290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:00.859458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:00.859633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] PATH_INFO: `/v3/projects/1e14773bfa4a4ed4901f4d1589b4a7b8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:00.862546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e14773bfa4a4ed4901f4d1589b4a7b8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:00.866740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:00.976895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:00.978458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:01.038972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e14773bfa4a4ed4901f4d1589b4a7b8', 'request_id': 'req-4b5fb371-2670-492f-a977-3ce8364d38ea'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:01.050841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e14773bfa4a4ed4901f4d1589b4a7b8', 'request_id': 'req-4b5fb371-2670-492f-a977-3ce8364d38ea'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:01.051752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67f074-ea16-4b2e-afce-85c084887c72 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:01.136721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e14773bfa4a4ed4901f4d1589b4a7b8', 'request_id': 'req-4b5fb371-2670-492f-a977-3ce8364d38ea'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:01.156249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3087bb-cb8f-4a4a-b1bf-2835e04d334c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:01.157777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3087bb-cb8f-4a4a-b1bf-2835e04d334c service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:01.158545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3087bb-cb8f-4a4a-b1bf-2835e04d334c service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:01.159290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3087bb-cb8f-4a4a-b1bf-2835e04d334c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:01.164684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:01.169923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ab3087bb-cb8f-4a4a-b1bf-2835e04d334c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:01.172664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] The token cache is being invalidate because project 1e14773bfa4a4ed4901f4d1589b4a7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:01.173436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:01.175098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4b5fb371-2670-492f-a977-3ce8364d38ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:01.178682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 991/1996] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:26:00 2022] DELETE /identity/v3/projects/1e14773bfa4a4ed4901f4d1589b4a7b8 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:01.185613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657ea0d-fd35-46a0-af59-928ce6c70def None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:01.186389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657ea0d-fd35-46a0-af59-928ce6c70def None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:01.187122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657ea0d-fd35-46a0-af59-928ce6c70def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:01.229967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0657ea0d-fd35-46a0-af59-928ce6c70def None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:01.231593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0657ea0d-fd35-46a0-af59-928ce6c70def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:01.253327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:01.286184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3087bb-cb8f-4a4a-b1bf-2835e04d334c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:01.292067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3087bb-cb8f-4a4a-b1bf-2835e04d334c service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:01.311807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1006/1997] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:26:01.370796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:01.382007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 992/1998] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:01.403089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c844af-043f-45ea-9b3f-e276938daa99 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:01.407599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c844af-043f-45ea-9b3f-e276938daa99 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:01.407599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c844af-043f-45ea-9b3f-e276938daa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:01.432055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a9c844af-043f-45ea-9b3f-e276938daa99 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:01.433668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:01.434622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a9c844af-043f-45ea-9b3f-e276938daa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:01.450808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1007/1999] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:01.451603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657ea0d-fd35-46a0-af59-928ce6c70def None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:01.462234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657ea0d-fd35-46a0-af59-928ce6c70def None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:01.472568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6734c2e-0c4f-4af2-8e27-0b63808f88c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:01.473422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6734c2e-0c4f-4af2-8e27-0b63808f88c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:01.473710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6734c2e-0c4f-4af2-8e27-0b63808f88c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:01.473994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6734c2e-0c4f-4af2-8e27-0b63808f88c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:01.474818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6734c2e-0c4f-4af2-8e27-0b63808f88c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:26:01.475372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6734c2e-0c4f-4af2-8e27-0b63808f88c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:01.477591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6734c2e-0c4f-4af2-8e27-0b63808f88c7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:01.493802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 993/2000] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:26:01.497981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c844af-043f-45ea-9b3f-e276938daa99 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:01.514235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c844af-043f-45ea-9b3f-e276938daa99 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:01.519988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211ba1c-1333-47b0-9bf8-e788ac1c5eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:01.520847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211ba1c-1333-47b0-9bf8-e788ac1c5eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:01.521137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211ba1c-1333-47b0-9bf8-e788ac1c5eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:01.521414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211ba1c-1333-47b0-9bf8-e788ac1c5eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:01.522141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211ba1c-1333-47b0-9bf8-e788ac1c5eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:01.524501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211ba1c-1333-47b0-9bf8-e788ac1c5eb9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:01.677662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1008/2001] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:26:01 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:26:01.682521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6734c2e-0c4f-4af2-8e27-0b63808f88c7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:01.703246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6734c2e-0c4f-4af2-8e27-0b63808f88c7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:01.776603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:01.816123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767405e8-0577-4c52-97fa-f9e55fb26842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:01.816828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767405e8-0577-4c52-97fa-f9e55fb26842 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:01.818080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767405e8-0577-4c52-97fa-f9e55fb26842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:01.818080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767405e8-0577-4c52-97fa-f9e55fb26842 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:01.818080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767405e8-0577-4c52-97fa-f9e55fb26842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:01.820394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767405e8-0577-4c52-97fa-f9e55fb26842 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:01.988179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211ba1c-1333-47b0-9bf8-e788ac1c5eb9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:01.990135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9211ba1c-1333-47b0-9bf8-e788ac1c5eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:02.016263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211ba1c-1333-47b0-9bf8-e788ac1c5eb9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:02.022162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:02.023390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:02.024035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:02.024472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] PATH_INFO: `/v3/users/0bcf2df379a44e4bb05c21c495c38483` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:02.033678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bcf2df379a44e4bb05c21c495c38483)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:02.033678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:02.075953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:02.088744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 994/2002] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:26:01 2022] POST /identity/v3/users => generated 287 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:02.090450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767405e8-0577-4c52-97fa-f9e55fb26842 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:02.109928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767405e8-0577-4c52-97fa-f9e55fb26842 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:02.119723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91be71-a60e-4227-8593-36d6b592e1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:02.121873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91be71-a60e-4227-8593-36d6b592e1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:02.130707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91be71-a60e-4227-8593-36d6b592e1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:02.130707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91be71-a60e-4227-8593-36d6b592e1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:02.130707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91be71-a60e-4227-8593-36d6b592e1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:02.130707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91be71-a60e-4227-8593-36d6b592e1ff admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:02.154219 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 995/2003] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:26:02.159634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91be71-a60e-4227-8593-36d6b592e1ff admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:02.174306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91be71-a60e-4227-8593-36d6b592e1ff admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:02.183189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfa857-ad5a-4969-852e-cc6dcd0ef249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:02.187931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfa857-ad5a-4969-852e-cc6dcd0ef249 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:02.187931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfa857-ad5a-4969-852e-cc6dcd0ef249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:02.187931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfa857-ad5a-4969-852e-cc6dcd0ef249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:02.187931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfa857-ad5a-4969-852e-cc6dcd0ef249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:02.187931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccfa857-ad5a-4969-852e-cc6dcd0ef249 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:02.198724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 996/2004] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:02.204279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfa857-ad5a-4969-852e-cc6dcd0ef249 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:02.216586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfa857-ad5a-4969-852e-cc6dcd0ef249 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:02.226162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228687f-6b4f-4536-9064-426afcd034bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:02.227200 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228687f-6b4f-4536-9064-426afcd034bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:02.227493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228687f-6b4f-4536-9064-426afcd034bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:02.233688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228687f-6b4f-4536-9064-426afcd034bb admin admin] PATH_INFO: `/v3/projects/0e2640459def4ed3b1716c7cac92a6a7/users/5763586297b2472ea3ed013da067ad0b/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:02.245392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228687f-6b4f-4536-9064-426afcd034bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2640459def4ed3b1716c7cac92a6a7, user_id=5763586297b2472ea3ed013da067ad0b, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:02.248278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228687f-6b4f-4536-9064-426afcd034bb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:02.352526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 997/2005] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:26:02 2022] PUT /identity/v3/projects/0e2640459def4ed3b1716c7cac92a6a7/users/5763586297b2472ea3ed013da067ad0b/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:26:02.360058 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58639e-d027-4339-a3b8-f60d97169c8d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:02.360441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58639e-d027-4339-a3b8-f60d97169c8d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:02.360834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58639e-d027-4339-a3b8-f60d97169c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:02.366447 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bcf2df379a44e4bb05c21c495c38483'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:02.379329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bcf2df379a44e4bb05c21c495c38483'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:02.380500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bcf2df379a44e4bb05c21c495c38483'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:02.381471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bcf2df379a44e4bb05c21c495c38483'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:02.384771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ba58639e-d027-4339-a3b8-f60d97169c8d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:02.386562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ba58639e-d027-4339-a3b8-f60d97169c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:02.506800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 998/2006] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:02.519832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bcf2df379a44e4bb05c21c495c38483'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:02.530312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58639e-d027-4339-a3b8-f60d97169c8d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:02.536534 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ba58639e-d027-4339-a3b8-f60d97169c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:02.557218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1009/2007] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:26:01 2022] DELETE /identity/v3/users/0bcf2df379a44e4bb05c21c495c38483 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:02.631022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58639e-d027-4339-a3b8-f60d97169c8d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:02.743459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1835f8-2380-43fc-98bc-77dee3593224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:02.744827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1835f8-2380-43fc-98bc-77dee3593224 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:02.745318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1835f8-2380-43fc-98bc-77dee3593224 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:02.745776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1835f8-2380-43fc-98bc-77dee3593224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:02.895386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1835f8-2380-43fc-98bc-77dee3593224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:02.896910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1835f8-2380-43fc-98bc-77dee3593224 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:02.909920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 999/2008] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:03.121278 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:03.123120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:03.146230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72945af0-3e53-47c9-aba0-9e110223f6f6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:03.153920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:03.156512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:03.156512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:03.157092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] PATH_INFO: `/v3/projects/99c98b6b528545f7a18638d5a0f69bb4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:03.159918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c98b6b528545f7a18638d5a0f69bb4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:03.162749 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:03.352232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c98b6b528545f7a18638d5a0f69bb4', 'request_id': 'req-d7cbba5a-56bd-444b-8deb-3f177be6f73f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:03.359513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c98b6b528545f7a18638d5a0f69bb4', 'request_id': 'req-d7cbba5a-56bd-444b-8deb-3f177be6f73f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:03.502972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c98b6b528545f7a18638d5a0f69bb4', 'request_id': 'req-d7cbba5a-56bd-444b-8deb-3f177be6f73f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:03.524433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] The token cache is being invalidate because project 99c98b6b528545f7a18638d5a0f69bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:03.550552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:03.550552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d7cbba5a-56bd-444b-8deb-3f177be6f73f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:03.550552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1010/2009] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:26:03 2022] DELETE /identity/v3/projects/99c98b6b528545f7a18638d5a0f69bb4 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:03.551437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773675d-0a99-44b7-9490-4348e9e97e41 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:03.551437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773675d-0a99-44b7-9490-4348e9e97e41 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:03.551437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773675d-0a99-44b7-9490-4348e9e97e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:03.571374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0773675d-0a99-44b7-9490-4348e9e97e41 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:03.572886 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0773675d-0a99-44b7-9490-4348e9e97e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:03.648290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1000/2010] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:03.657485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb893ec-48c7-4f27-b9c4-2dfaa9400f26 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:03.657851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb893ec-48c7-4f27-b9c4-2dfaa9400f26 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:03.658135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb893ec-48c7-4f27-b9c4-2dfaa9400f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:03.679024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5bb893ec-48c7-4f27-b9c4-2dfaa9400f26 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:03.680184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5bb893ec-48c7-4f27-b9c4-2dfaa9400f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:03.691257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1011/2011] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:03.697133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773675d-0a99-44b7-9490-4348e9e97e41 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:03.710309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773675d-0a99-44b7-9490-4348e9e97e41 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:03.713738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2769729-5226-432e-b22e-0d2abf3a9c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:03.714447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2769729-5226-432e-b22e-0d2abf3a9c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:03.714739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2769729-5226-432e-b22e-0d2abf3a9c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:03.715019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2769729-5226-432e-b22e-0d2abf3a9c33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:03.715772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2769729-5226-432e-b22e-0d2abf3a9c33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:26:03.716082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2769729-5226-432e-b22e-0d2abf3a9c33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:03.717787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2769729-5226-432e-b22e-0d2abf3a9c33 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:03.730564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1001/2012] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:26:03.735664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb893ec-48c7-4f27-b9c4-2dfaa9400f26 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:03.745480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb893ec-48c7-4f27-b9c4-2dfaa9400f26 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:03.751429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68142d-d1b3-4e8e-b50d-056036baf951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:03.752116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68142d-d1b3-4e8e-b50d-056036baf951 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:03.752404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68142d-d1b3-4e8e-b50d-056036baf951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:03.752766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68142d-d1b3-4e8e-b50d-056036baf951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:03.753457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68142d-d1b3-4e8e-b50d-056036baf951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:03.755449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68142d-d1b3-4e8e-b50d-056036baf951 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:03.990733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1012/2013] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:26:03 2022] POST /identity/v3/projects => generated 383 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:26:03.996519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2769729-5226-432e-b22e-0d2abf3a9c33 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:04.011744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2769729-5226-432e-b22e-0d2abf3a9c33 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:04.134318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a1067-7584-43d6-a7a7-d7133bdabd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:04.135053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a1067-7584-43d6-a7a7-d7133bdabd2c admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:04.135444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a1067-7584-43d6-a7a7-d7133bdabd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:04.135610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a1067-7584-43d6-a7a7-d7133bdabd2c admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:04.136328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a1067-7584-43d6-a7a7-d7133bdabd2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:04.140088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a1067-7584-43d6-a7a7-d7133bdabd2c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:04.194798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68142d-d1b3-4e8e-b50d-056036baf951 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:04.198604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3e68142d-d1b3-4e8e-b50d-056036baf951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:04.249980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1002/2014] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:26:03 2022] POST /identity/v3/users => generated 292 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:04.254101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a1067-7584-43d6-a7a7-d7133bdabd2c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:04.265531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a1067-7584-43d6-a7a7-d7133bdabd2c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:04.272062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eca5c-a7f8-4243-abb7-8f638fdd2d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:04.275463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eca5c-a7f8-4243-abb7-8f638fdd2d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:04.276122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eca5c-a7f8-4243-abb7-8f638fdd2d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:04.277073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eca5c-a7f8-4243-abb7-8f638fdd2d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:04.277937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eca5c-a7f8-4243-abb7-8f638fdd2d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:04.279819 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eca5c-a7f8-4243-abb7-8f638fdd2d40 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:04.292852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1003/2015] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:26:04.299029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eca5c-a7f8-4243-abb7-8f638fdd2d40 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:04.313679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eca5c-a7f8-4243-abb7-8f638fdd2d40 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:04.321949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff1b59-51cb-4a9d-836a-b655685a3bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:04.323143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff1b59-51cb-4a9d-836a-b655685a3bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:04.323626 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff1b59-51cb-4a9d-836a-b655685a3bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:04.324142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff1b59-51cb-4a9d-836a-b655685a3bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:04.325923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff1b59-51cb-4a9d-836a-b655685a3bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:04.327755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68142d-d1b3-4e8e-b50d-056036baf951 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:04.328729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff1b59-51cb-4a9d-836a-b655685a3bb0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:04.335710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44c0e2-a5f5-46e6-87af-fcc91fe80f30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:04.336671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44c0e2-a5f5-46e6-87af-fcc91fe80f30 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:04.337025 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44c0e2-a5f5-46e6-87af-fcc91fe80f30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:04.337385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44c0e2-a5f5-46e6-87af-fcc91fe80f30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:04.346152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5f44c0e2-a5f5-46e6-87af-fcc91fe80f30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:04.346360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1004/2016] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:26:04.353126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff1b59-51cb-4a9d-836a-b655685a3bb0 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:04.374286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff1b59-51cb-4a9d-836a-b655685a3bb0 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:04.375277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:04.382313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b298d-fc1a-4041-bfc4-ddb4d18b5c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:04.383939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b298d-fc1a-4041-bfc4-ddb4d18b5c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:04.384424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b298d-fc1a-4041-bfc4-ddb4d18b5c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:04.384752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b298d-fc1a-4041-bfc4-ddb4d18b5c4c admin admin] PATH_INFO: `/v3/projects/350cba15727143359613e4f1369b8efd/users/4741fd44d9b2449fb3e40bccae27d84f/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:04.436770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b298d-fc1a-4041-bfc4-ddb4d18b5c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=350cba15727143359613e4f1369b8efd, user_id=4741fd44d9b2449fb3e40bccae27d84f, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:04.445436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b298d-fc1a-4041-bfc4-ddb4d18b5c4c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:04.446622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44c0e2-a5f5-46e6-87af-fcc91fe80f30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:04.448251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44c0e2-a5f5-46e6-87af-fcc91fe80f30 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:04.462951 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1013/2017] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:26:04.485136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e44d9-60cb-46ef-920d-68e6415d9a76 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:04.485409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e44d9-60cb-46ef-920d-68e6415d9a76 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:04.486039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e44d9-60cb-46ef-920d-68e6415d9a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:04.509582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d81e44d9-60cb-46ef-920d-68e6415d9a76 None None] MFA Rules not processed for user `7caa126f7eab4eef8db03374e82a6ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:04.511009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d81e44d9-60cb-46ef-920d-68e6415d9a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:04.589460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1014/2018] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:04.626375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1005/2019] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:26:04 2022] PUT /identity/v3/projects/350cba15727143359613e4f1369b8efd/users/4741fd44d9b2449fb3e40bccae27d84f/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:04.644601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e34e96-be06-468f-b8bd-879ac98e5c91 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:04.645422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e34e96-be06-468f-b8bd-879ac98e5c91 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:04.645946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e34e96-be06-468f-b8bd-879ac98e5c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:04.696731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-66e34e96-be06-468f-b8bd-879ac98e5c91 None None] MFA Rules not processed for user `4741fd44d9b2449fb3e40bccae27d84f`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:04.699275 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-66e34e96-be06-468f-b8bd-879ac98e5c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:04.797782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1015/2020] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:04.816356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b298d-fc1a-4041-bfc4-ddb4d18b5c4c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:04.818163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2e9b298d-fc1a-4041-bfc4-ddb4d18b5c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:04.952125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b298d-fc1a-4041-bfc4-ddb4d18b5c4c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:04.960764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52be13-a48c-401b-8b43-57ba3df2cfdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:04.961447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52be13-a48c-401b-8b43-57ba3df2cfdd service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:04.961736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52be13-a48c-401b-8b43-57ba3df2cfdd service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:04.962018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52be13-a48c-401b-8b43-57ba3df2cfdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:04.972373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52be13-a48c-401b-8b43-57ba3df2cfdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:04.974160 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52be13-a48c-401b-8b43-57ba3df2cfdd service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:04.987237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1006/2021] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:26:05.058891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:05.348191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e712fc-c243-4ebe-a036-f3e55db22ae2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:05.350019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e712fc-c243-4ebe-a036-f3e55db22ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:05.350019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e712fc-c243-4ebe-a036-f3e55db22ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:05.400984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b6e712fc-c243-4ebe-a036-f3e55db22ae2 None None] MFA Rules not processed for user `5763586297b2472ea3ed013da067ad0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:05.403098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b6e712fc-c243-4ebe-a036-f3e55db22ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:05.541213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1016/2022] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:05.551016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52be13-a48c-401b-8b43-57ba3df2cfdd service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:05.560889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2b52be13-a48c-401b-8b43-57ba3df2cfdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:05.728545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52be13-a48c-401b-8b43-57ba3df2cfdd service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:05.738513 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1f8ce-7946-4220-aeb2-6d2f5c1f91b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:05.739424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1f8ce-7946-4220-aeb2-6d2f5c1f91b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:05.740026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1f8ce-7946-4220-aeb2-6d2f5c1f91b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:05.740700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1f8ce-7946-4220-aeb2-6d2f5c1f91b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:05.741983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1f8ce-7946-4220-aeb2-6d2f5c1f91b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:05.745596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1f8ce-7946-4220-aeb2-6d2f5c1f91b8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:05.901415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1007/2023] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:26:05 2022] POST /identity/v3/projects => generated 373 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:26:05.908696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e712fc-c243-4ebe-a036-f3e55db22ae2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:05.925894 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e712fc-c243-4ebe-a036-f3e55db22ae2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:06.038083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d47a39-8298-4f62-b1d5-cf4c5cb83739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:06.039398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d47a39-8298-4f62-b1d5-cf4c5cb83739 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:06.039986 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d47a39-8298-4f62-b1d5-cf4c5cb83739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:06.040527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d47a39-8298-4f62-b1d5-cf4c5cb83739 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:06.042350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d47a39-8298-4f62-b1d5-cf4c5cb83739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:06.046948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d47a39-8298-4f62-b1d5-cf4c5cb83739 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:06.187410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1017/2024] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:26:05 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:06.192136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1f8ce-7946-4220-aeb2-6d2f5c1f91b8 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:06.204591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1f8ce-7946-4220-aeb2-6d2f5c1f91b8 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:06.210970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84905cd5-e2cd-4ce6-86a8-f3f5a3885eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:06.211779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84905cd5-e2cd-4ce6-86a8-f3f5a3885eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:06.212133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84905cd5-e2cd-4ce6-86a8-f3f5a3885eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:06.214526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84905cd5-e2cd-4ce6-86a8-f3f5a3885eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:06.214526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84905cd5-e2cd-4ce6-86a8-f3f5a3885eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:06.215571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84905cd5-e2cd-4ce6-86a8-f3f5a3885eb7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:06.230895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1008/2025] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:06.233386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d47a39-8298-4f62-b1d5-cf4c5cb83739 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:06.246339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d47a39-8298-4f62-b1d5-cf4c5cb83739 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:06.256915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7f1b1-ac07-4c06-95b6-99bed34d5fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:06.257901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f1b1-ac07-4c06-95b6-99bed34d5fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:06.258195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f1b1-ac07-4c06-95b6-99bed34d5fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:06.260443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f1b1-ac07-4c06-95b6-99bed34d5fb5 admin admin] PATH_INFO: `/v3/projects/e2d0451cc99743a9a3e600fe30b7b4be/users/f4ef72893291484f9d623680e1b54901/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:06.278211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7f1b1-ac07-4c06-95b6-99bed34d5fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d0451cc99743a9a3e600fe30b7b4be, user_id=f4ef72893291484f9d623680e1b54901, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:06.281719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7f1b1-ac07-4c06-95b6-99bed34d5fb5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:06.434860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1018/2026] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:26:06 2022] PUT /identity/v3/projects/e2d0451cc99743a9a3e600fe30b7b4be/users/f4ef72893291484f9d623680e1b54901/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:06.441180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84905cd5-e2cd-4ce6-86a8-f3f5a3885eb7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:06.463093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84905cd5-e2cd-4ce6-86a8-f3f5a3885eb7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:06.540005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbdb2d-bbd1-4eed-b1bc-a9762884833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:06.540787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbdb2d-bbd1-4eed-b1bc-a9762884833f admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:06.541179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbdb2d-bbd1-4eed-b1bc-a9762884833f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:06.541503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbdb2d-bbd1-4eed-b1bc-a9762884833f admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:06.542345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbdb2d-bbd1-4eed-b1bc-a9762884833f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:06.544106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbdb2d-bbd1-4eed-b1bc-a9762884833f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:06.557046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1009/2027] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:26:06.563466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7f1b1-ac07-4c06-95b6-99bed34d5fb5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:06.574890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7f1b1-ac07-4c06-95b6-99bed34d5fb5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:06.580216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdd4f0-773c-4507-a48e-c5ef3c4b8512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:06.581543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdd4f0-773c-4507-a48e-c5ef3c4b8512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:06.581834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdd4f0-773c-4507-a48e-c5ef3c4b8512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:06.582104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdd4f0-773c-4507-a48e-c5ef3c4b8512 admin admin] PATH_INFO: `/v3/projects/e2d0451cc99743a9a3e600fe30b7b4be/users/f4ef72893291484f9d623680e1b54901/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:06.584483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdd4f0-773c-4507-a48e-c5ef3c4b8512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d0451cc99743a9a3e600fe30b7b4be, user_id=f4ef72893291484f9d623680e1b54901, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:06.587483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdd4f0-773c-4507-a48e-c5ef3c4b8512 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:06.600868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1019/2028] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:26:06 2022] PUT /identity/v3/projects/e2d0451cc99743a9a3e600fe30b7b4be/users/f4ef72893291484f9d623680e1b54901/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:07.268386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=15 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:08.002807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:08.805076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaebd6-66a9-426e-b6cc-48a7c57d48eb None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:08.805458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaebd6-66a9-426e-b6cc-48a7c57d48eb None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:08.805752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaebd6-66a9-426e-b6cc-48a7c57d48eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:08.846600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-afaaebd6-66a9-426e-b6cc-48a7c57d48eb None None] MFA Rules not processed for user `f4ef72893291484f9d623680e1b54901`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:08.847834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-afaaebd6-66a9-426e-b6cc-48a7c57d48eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:08.951148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1010/2029] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:08.983612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdd4f0-773c-4507-a48e-c5ef3c4b8512 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:08.985940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-97cdd4f0-773c-4507-a48e-c5ef3c4b8512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:09.093145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdd4f0-773c-4507-a48e-c5ef3c4b8512 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:09.097581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5a9ff-fc52-4d5a-874e-946a57c6a23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:09.098301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5a9ff-fc52-4d5a-874e-946a57c6a23e service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:09.098629 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5a9ff-fc52-4d5a-874e-946a57c6a23e service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:09.098919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5a9ff-fc52-4d5a-874e-946a57c6a23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:09.204244 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5a9ff-fc52-4d5a-874e-946a57c6a23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:09.206114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5a9ff-fc52-4d5a-874e-946a57c6a23e service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:09.221379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1020/2030] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:26:09.738150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:09.903982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:11.182398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:11.861893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:16.784528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaebd6-66a9-426e-b6cc-48a7c57d48eb None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:16.799931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaebd6-66a9-426e-b6cc-48a7c57d48eb None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:16.804782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1794e30-4e61-4584-adf6-b9730962b7c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:16.807068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794e30-4e61-4584-adf6-b9730962b7c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:16.807068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794e30-4e61-4584-adf6-b9730962b7c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:16.807068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794e30-4e61-4584-adf6-b9730962b7c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:16.824491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1794e30-4e61-4584-adf6-b9730962b7c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:16.827567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1794e30-4e61-4584-adf6-b9730962b7c5 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:16.849267 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1011/2031] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:17.085117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5a9ff-fc52-4d5a-874e-946a57c6a23e service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:17.086938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bed5a9ff-fc52-4d5a-874e-946a57c6a23e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:17.190010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5a9ff-fc52-4d5a-874e-946a57c6a23e service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:17.195950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:17.196866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:17.197149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:17.197423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] PATH_INFO: `/v3/users/5b22ebc6ae074c8fb1fedc593a3b6ab0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:17.198834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b22ebc6ae074c8fb1fedc593a3b6ab0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:17.200804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:17.345986 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:17.464547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b22ebc6ae074c8fb1fedc593a3b6ab0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:17.478728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b22ebc6ae074c8fb1fedc593a3b6ab0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:17.479560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b22ebc6ae074c8fb1fedc593a3b6ab0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:17.480210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b22ebc6ae074c8fb1fedc593a3b6ab0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:17.582360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b22ebc6ae074c8fb1fedc593a3b6ab0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:17.606548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1021/2032] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:26:17 2022] DELETE /identity/v3/users/5b22ebc6ae074c8fb1fedc593a3b6ab0 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:17.612152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd25fcc-70b6-4c0e-a88e-51eb467f5304 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:17.612442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd25fcc-70b6-4c0e-a88e-51eb467f5304 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:17.613432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd25fcc-70b6-4c0e-a88e-51eb467f5304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:17.636369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9dd25fcc-70b6-4c0e-a88e-51eb467f5304 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:17.638357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9dd25fcc-70b6-4c0e-a88e-51eb467f5304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:17.743655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1012/2033] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:17.758510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:17.760872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:17.873865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7db86-4a39-4317-aec7-dacec9a01d28 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:17.879283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1407db-fcf5-4095-bdb1-df6462cd510f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:17.880459 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1407db-fcf5-4095-bdb1-df6462cd510f service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:17.880973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1407db-fcf5-4095-bdb1-df6462cd510f service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:17.881478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1407db-fcf5-4095-bdb1-df6462cd510f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:17.894360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1407db-fcf5-4095-bdb1-df6462cd510f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:17.895906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1407db-fcf5-4095-bdb1-df6462cd510f service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:17.909434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1022/2034] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:18.585379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd25fcc-70b6-4c0e-a88e-51eb467f5304 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:18.588776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9dd25fcc-70b6-4c0e-a88e-51eb467f5304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:18.616107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd25fcc-70b6-4c0e-a88e-51eb467f5304 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:18.625796 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:18.627248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:18.627991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:18.628609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] PATH_INFO: `/v3/projects/eec0810c2e354026a983ca1f586c33c1` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:18.630283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec0810c2e354026a983ca1f586c33c1)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:18.632448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:18.789683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec0810c2e354026a983ca1f586c33c1', 'request_id': 'req-08dab05d-988f-4a9f-ad93-3239a87870f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:18.798522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec0810c2e354026a983ca1f586c33c1', 'request_id': 'req-08dab05d-988f-4a9f-ad93-3239a87870f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:18.802148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:18.921333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec0810c2e354026a983ca1f586c33c1', 'request_id': 'req-08dab05d-988f-4a9f-ad93-3239a87870f4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:18.938546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] The token cache is being invalidate because project eec0810c2e354026a983ca1f586c33c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:18.938878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:18.939601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:18.941662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1013/2035] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:26:18 2022] DELETE /identity/v3/projects/eec0810c2e354026a983ca1f586c33c1 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:18.947369 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1407db-fcf5-4095-bdb1-df6462cd510f service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:18.962376 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1407db-fcf5-4095-bdb1-df6462cd510f service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:19.074037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:19.079383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:19.079870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:19.080375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] PATH_INFO: `/v3/users/82dade18aa994c2b948a0c520c9db619` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:19.085901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=82dade18aa994c2b948a0c520c9db619)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:19.099358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:19.249158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:19.249978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:19.283262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dab05d-988f-4a9f-ad93-3239a87870f4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:19.317433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:19.318959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:19.330090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:19.330821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] PATH_INFO: `/v3/users/4741fd44d9b2449fb3e40bccae27d84f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:19.335116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4741fd44d9b2449fb3e40bccae27d84f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:19.346075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:20.043942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82dade18aa994c2b948a0c520c9db619'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.046301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:20.053966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82dade18aa994c2b948a0c520c9db619'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.054879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82dade18aa994c2b948a0c520c9db619'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.055606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82dade18aa994c2b948a0c520c9db619'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.195154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4741fd44d9b2449fb3e40bccae27d84f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.195929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4741fd44d9b2449fb3e40bccae27d84f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.211021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4741fd44d9b2449fb3e40bccae27d84f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.211914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4741fd44d9b2449fb3e40bccae27d84f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.308790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0896983b-8dbb-43e0-ab90-895f4b986edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82dade18aa994c2b948a0c520c9db619'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.331964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1023/2036] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:26:18 2022] DELETE /identity/v3/users/82dade18aa994c2b948a0c520c9db619 => generated 0 bytes in 1385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:20.414312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4741fd44d9b2449fb3e40bccae27d84f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:20.432258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1014/2037] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:26:19 2022] DELETE /identity/v3/users/4741fd44d9b2449fb3e40bccae27d84f => generated 0 bytes in 1186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:20.440446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e7063-6d27-4047-84a7-bb49fd1b8811 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:20.441328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e7063-6d27-4047-84a7-bb49fd1b8811 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:20.441818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e7063-6d27-4047-84a7-bb49fd1b8811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:20.476237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-829e7063-6d27-4047-84a7-bb49fd1b8811 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:20.477710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-829e7063-6d27-4047-84a7-bb49fd1b8811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:20.626713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1024/2038] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:20.640965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:20.642799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:20.744373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93c149-e2d2-43e3-9994-b18f5ad7f9d6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:20.758980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c552eb-d699-4c54-bbb8-ddbfe1b38afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:20.760556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c552eb-d699-4c54-bbb8-ddbfe1b38afc service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:20.761054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c552eb-d699-4c54-bbb8-ddbfe1b38afc service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:20.761564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c552eb-d699-4c54-bbb8-ddbfe1b38afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:20.782717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c552eb-d699-4c54-bbb8-ddbfe1b38afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:20.787301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c552eb-d699-4c54-bbb8-ddbfe1b38afc service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:20.808679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1015/2039] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:26:21.062607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e7063-6d27-4047-84a7-bb49fd1b8811 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:21.062607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-829e7063-6d27-4047-84a7-bb49fd1b8811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:21.083735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e7063-6d27-4047-84a7-bb49fd1b8811 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:21.092726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:21.097127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:21.097127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:21.097127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] PATH_INFO: `/v3/projects/c2ca676e92db4f8e8d3db5da953e6045` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:21.097741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ca676e92db4f8e8d3db5da953e6045)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:21.103725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:21.289358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ca676e92db4f8e8d3db5da953e6045', 'request_id': 'req-eebaf404-3c5a-4086-8358-6ae135f72129'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.297783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ca676e92db4f8e8d3db5da953e6045', 'request_id': 'req-eebaf404-3c5a-4086-8358-6ae135f72129'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.387958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ca676e92db4f8e8d3db5da953e6045', 'request_id': 'req-eebaf404-3c5a-4086-8358-6ae135f72129'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.412472 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] The token cache is being invalidate because project c2ca676e92db4f8e8d3db5da953e6045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:21.413485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.414293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.420896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1025/2040] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:26:21 2022] DELETE /identity/v3/projects/c2ca676e92db4f8e8d3db5da953e6045 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:21.426832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c552eb-d699-4c54-bbb8-ddbfe1b38afc service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:21.440590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c552eb-d699-4c54-bbb8-ddbfe1b38afc service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:21.512087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:21.514495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:21.543262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:21.545656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:21.546355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:21.547051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] PATH_INFO: `/v3/users/c914887aa194403f8c80660662a0f1ad` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:21.551136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c914887aa194403f8c80660662a0f1ad)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:21.553340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:21.563747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:21.631403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:21.667661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaf404-3c5a-4086-8358-6ae135f72129 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:21.679835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:21.681466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:21.685460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:21.685460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] PATH_INFO: `/v3/projects/350cba15727143359613e4f1369b8efd` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:21.685460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] RBAC: Authorizing `identity:delete_project(project_id=350cba15727143359613e4f1369b8efd)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:21.690033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:21.882011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350cba15727143359613e4f1369b8efd', 'request_id': 'req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.882910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c914887aa194403f8c80660662a0f1ad'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.884024 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c914887aa194403f8c80660662a0f1ad'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.890673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350cba15727143359613e4f1369b8efd', 'request_id': 'req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.908916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c914887aa194403f8c80660662a0f1ad'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.910083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c914887aa194403f8c80660662a0f1ad'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:21.976352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350cba15727143359613e4f1369b8efd', 'request_id': 'req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:22.001878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] The token cache is being invalidate because project 350cba15727143359613e4f1369b8efd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:22.002075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:22.002712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f101b17-7cd0-4c01-b43d-e410c7b6b27a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:22.004878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1026/2041] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:26:21 2022] DELETE /identity/v3/projects/350cba15727143359613e4f1369b8efd => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:26:22.012272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380648bf-fa8b-4e96-bc99-62a625f40e19 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:22.012610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380648bf-fa8b-4e96-bc99-62a625f40e19 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:22.012928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380648bf-fa8b-4e96-bc99-62a625f40e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:22.053773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-380648bf-fa8b-4e96-bc99-62a625f40e19 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:22.056026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-380648bf-fa8b-4e96-bc99-62a625f40e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:22.126772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7248e513-b80f-4a44-81f9-186b48ef4acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c914887aa194403f8c80660662a0f1ad'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:22.152433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1016/2042] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:26:21 2022] DELETE /identity/v3/users/c914887aa194403f8c80660662a0f1ad => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:22.253017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1027/2043] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:26:22.271543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefec07-7071-4358-8295-25d5a3a2016d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:22.272209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefec07-7071-4358-8295-25d5a3a2016d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:22.272697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefec07-7071-4358-8295-25d5a3a2016d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:22.308362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0eefec07-7071-4358-8295-25d5a3a2016d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:22.312099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0eefec07-7071-4358-8295-25d5a3a2016d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:22.454433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1017/2044] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:22.455816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380648bf-fa8b-4e96-bc99-62a625f40e19 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:22.498890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380648bf-fa8b-4e96-bc99-62a625f40e19 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:22.516984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229e889-de0b-4171-9467-36a4b19bac2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:22.518071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229e889-de0b-4171-9467-36a4b19bac2c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:22.518532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229e889-de0b-4171-9467-36a4b19bac2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:22.518851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229e889-de0b-4171-9467-36a4b19bac2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:22.524944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229e889-de0b-4171-9467-36a4b19bac2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:26:22.528711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229e889-de0b-4171-9467-36a4b19bac2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:22.538197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229e889-de0b-4171-9467-36a4b19bac2c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:22.582315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1028/2045] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:26:22.594334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefec07-7071-4358-8295-25d5a3a2016d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:22.619657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefec07-7071-4358-8295-25d5a3a2016d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:22.635079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccdb7df-d326-4fca-a9fa-285d746077dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:22.635518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdb7df-d326-4fca-a9fa-285d746077dd admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:22.636354 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdb7df-d326-4fca-a9fa-285d746077dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:22.636948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdb7df-d326-4fca-a9fa-285d746077dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:22.638440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccdb7df-d326-4fca-a9fa-285d746077dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:22.647727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccdb7df-d326-4fca-a9fa-285d746077dd admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:22.822489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1018/2046] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:26:22 2022] POST /identity/v3/projects => generated 357 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:22.827412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229e889-de0b-4171-9467-36a4b19bac2c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:22.843862 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229e889-de0b-4171-9467-36a4b19bac2c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:22.954898 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141261c2-9e5b-4f64-858a-a46ebd57c222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:22.955958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141261c2-9e5b-4f64-858a-a46ebd57c222 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:22.956449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141261c2-9e5b-4f64-858a-a46ebd57c222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:22.956903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141261c2-9e5b-4f64-858a-a46ebd57c222 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:22.958179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141261c2-9e5b-4f64-858a-a46ebd57c222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:22.962336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141261c2-9e5b-4f64-858a-a46ebd57c222 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:22.984884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:22.985061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccdb7df-d326-4fca-a9fa-285d746077dd admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:22.988151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1ccdb7df-d326-4fca-a9fa-285d746077dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:23.010881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccdb7df-d326-4fca-a9fa-285d746077dd admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:23.021388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:23.031692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:23.031692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:23.031692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] PATH_INFO: `/v3/projects/86cce31d941f4c9cbc8c9769899403c0` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:23.031692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=86cce31d941f4c9cbc8c9769899403c0)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:23.032256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:23.094950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1029/2047] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:26:22 2022] POST /identity/v3/users => generated 279 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:23.100231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141261c2-9e5b-4f64-858a-a46ebd57c222 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:23.115979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141261c2-9e5b-4f64-858a-a46ebd57c222 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:23.121124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a9109-bfa9-447e-a069-399612fee7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:23.121809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a9109-bfa9-447e-a069-399612fee7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:23.122095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a9109-bfa9-447e-a069-399612fee7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:23.122371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a9109-bfa9-447e-a069-399612fee7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:23.123511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a9109-bfa9-447e-a069-399612fee7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:23.125207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a9109-bfa9-447e-a069-399612fee7c6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:23.139078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1030/2048] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:23.147222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a9109-bfa9-447e-a069-399612fee7c6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:23.159069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a9109-bfa9-447e-a069-399612fee7c6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:23.164187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bac5f-a7fc-4465-a025-d04079b7710c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:23.164877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bac5f-a7fc-4465-a025-d04079b7710c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:23.165191 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bac5f-a7fc-4465-a025-d04079b7710c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:23.165427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bac5f-a7fc-4465-a025-d04079b7710c admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:23.166164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784bac5f-a7fc-4465-a025-d04079b7710c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:23.167837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784bac5f-a7fc-4465-a025-d04079b7710c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:23.183557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1031/2049] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:26:23.189285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bac5f-a7fc-4465-a025-d04079b7710c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:23.204045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bac5f-a7fc-4465-a025-d04079b7710c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:23.211313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86cce31d941f4c9cbc8c9769899403c0', 'request_id': 'req-37d81956-ad42-4e06-822d-a394842104ef'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:23.228822 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:23.230631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86cce31d941f4c9cbc8c9769899403c0', 'request_id': 'req-37d81956-ad42-4e06-822d-a394842104ef'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:23.316209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8433fc-9b8a-446d-b168-3c7f9ac00208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:23.317675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8433fc-9b8a-446d-b168-3c7f9ac00208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:23.318164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8433fc-9b8a-446d-b168-3c7f9ac00208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:23.318912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8433fc-9b8a-446d-b168-3c7f9ac00208 admin admin] PATH_INFO: `/v3/projects/6b4cbc799d504116a0666915febb872c/users/fc642bf43bb24b7b8c10b2d675b0fea5/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:23.330380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86cce31d941f4c9cbc8c9769899403c0', 'request_id': 'req-37d81956-ad42-4e06-822d-a394842104ef'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:23.344985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] The token cache is being invalidate because project 86cce31d941f4c9cbc8c9769899403c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:23.345277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:23.345991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-37d81956-ad42-4e06-822d-a394842104ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:23.347818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1019/2050] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:26:22 2022] DELETE /identity/v3/projects/86cce31d941f4c9cbc8c9769899403c0 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:23.349755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8433fc-9b8a-446d-b168-3c7f9ac00208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4cbc799d504116a0666915febb872c, user_id=fc642bf43bb24b7b8c10b2d675b0fea5, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:23.361997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8433fc-9b8a-446d-b168-3c7f9ac00208 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:23.492433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1032/2051] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:26:23 2022] PUT /identity/v3/projects/6b4cbc799d504116a0666915febb872c/users/fc642bf43bb24b7b8c10b2d675b0fea5/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:26:23.504916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333db459-19d6-447d-b5c0-765f6d9427c9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:23.505987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333db459-19d6-447d-b5c0-765f6d9427c9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:23.505987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333db459-19d6-447d-b5c0-765f6d9427c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:23.544307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-333db459-19d6-447d-b5c0-765f6d9427c9 None None] MFA Rules not processed for user `fc642bf43bb24b7b8c10b2d675b0fea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:23.547665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-333db459-19d6-447d-b5c0-765f6d9427c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:23.740937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1020/2052] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:23 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:23.749382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8433fc-9b8a-446d-b168-3c7f9ac00208 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:23.750997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6b8433fc-9b8a-446d-b168-3c7f9ac00208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:23.883353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8433fc-9b8a-446d-b168-3c7f9ac00208 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:23.891743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e8faf-ccab-4159-a7c9-fb5e752c3475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:23.893528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e8faf-ccab-4159-a7c9-fb5e752c3475 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:23.893528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e8faf-ccab-4159-a7c9-fb5e752c3475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:23.894039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e8faf-ccab-4159-a7c9-fb5e752c3475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:23.895526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e8faf-ccab-4159-a7c9-fb5e752c3475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:23.903842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e8faf-ccab-4159-a7c9-fb5e752c3475 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:24.104554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333db459-19d6-447d-b5c0-765f6d9427c9 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:24.104859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1033/2053] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:26:23 2022] POST /identity/v3/projects => generated 355 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:24.111410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333db459-19d6-447d-b5c0-765f6d9427c9 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:24.199373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaf5c2-3cc8-4287-8de3-331e2b4689e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:24.200810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaf5c2-3cc8-4287-8de3-331e2b4689e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:24.201623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaf5c2-3cc8-4287-8de3-331e2b4689e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:24.202152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaf5c2-3cc8-4287-8de3-331e2b4689e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:24.203472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eaf5c2-3cc8-4287-8de3-331e2b4689e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:24.206368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eaf5c2-3cc8-4287-8de3-331e2b4689e7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:24.336307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1021/2054] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:26:24 2022] POST /identity/v3/users => generated 284 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:26:24.340643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e8faf-ccab-4159-a7c9-fb5e752c3475 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:24.352968 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e8faf-ccab-4159-a7c9-fb5e752c3475 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:24.358657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b134b-b1f5-4cf8-92bd-286ee022bd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:24.359364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b134b-b1f5-4cf8-92bd-286ee022bd02 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:24.359663 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b134b-b1f5-4cf8-92bd-286ee022bd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:24.359945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b134b-b1f5-4cf8-92bd-286ee022bd02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:24.360692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b134b-b1f5-4cf8-92bd-286ee022bd02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:24.362329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b134b-b1f5-4cf8-92bd-286ee022bd02 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:24.375778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1034/2055] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:24.382264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaf5c2-3cc8-4287-8de3-331e2b4689e7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:24.400087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaf5c2-3cc8-4287-8de3-331e2b4689e7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:24.408276 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809b095-a68c-453c-bf54-0215a4dcb751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:24.409774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809b095-a68c-453c-bf54-0215a4dcb751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:24.410215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809b095-a68c-453c-bf54-0215a4dcb751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:24.410697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d809b095-a68c-453c-bf54-0215a4dcb751 admin admin] PATH_INFO: `/v3/projects/a8818307692d498db2541fb2a6dcc6d1/users/cbb5e189a15d4eaa8e42c205e0e6bec5/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:24.439480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809b095-a68c-453c-bf54-0215a4dcb751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8818307692d498db2541fb2a6dcc6d1, user_id=cbb5e189a15d4eaa8e42c205e0e6bec5, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:24.444587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d809b095-a68c-453c-bf54-0215a4dcb751 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:24.615222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1022/2056] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:26:24 2022] PUT /identity/v3/projects/a8818307692d498db2541fb2a6dcc6d1/users/cbb5e189a15d4eaa8e42c205e0e6bec5/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:24.619748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b134b-b1f5-4cf8-92bd-286ee022bd02 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:24.631770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b134b-b1f5-4cf8-92bd-286ee022bd02 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:24.738814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd7fe6-1ce8-4408-9356-e721af5c9c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:24.743519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd7fe6-1ce8-4408-9356-e721af5c9c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:24.743519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd7fe6-1ce8-4408-9356-e721af5c9c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:24.743519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd7fe6-1ce8-4408-9356-e721af5c9c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:24.743519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd7fe6-1ce8-4408-9356-e721af5c9c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:24.743519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd7fe6-1ce8-4408-9356-e721af5c9c73 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:24.757939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1035/2057] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:26:24.762749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809b095-a68c-453c-bf54-0215a4dcb751 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:24.776674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d809b095-a68c-453c-bf54-0215a4dcb751 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:24.782148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595dddd-4dd7-4b11-b807-8949e1904437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:24.783103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595dddd-4dd7-4b11-b807-8949e1904437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:24.783397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595dddd-4dd7-4b11-b807-8949e1904437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:24.783670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595dddd-4dd7-4b11-b807-8949e1904437 admin admin] PATH_INFO: `/v3/projects/a8818307692d498db2541fb2a6dcc6d1/users/cbb5e189a15d4eaa8e42c205e0e6bec5/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:24.786332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595dddd-4dd7-4b11-b807-8949e1904437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8818307692d498db2541fb2a6dcc6d1, user_id=cbb5e189a15d4eaa8e42c205e0e6bec5, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:24.789099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595dddd-4dd7-4b11-b807-8949e1904437 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:24.809175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1023/2058] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:26:24 2022] PUT /identity/v3/projects/a8818307692d498db2541fb2a6dcc6d1/users/cbb5e189a15d4eaa8e42c205e0e6bec5/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:26:24.829689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b1945-376f-4c8a-8d0a-1d0219f2773d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:24.830199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b1945-376f-4c8a-8d0a-1d0219f2773d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:24.830777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b1945-376f-4c8a-8d0a-1d0219f2773d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:24.875636 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c97b1945-376f-4c8a-8d0a-1d0219f2773d None None] MFA Rules not processed for user `cbb5e189a15d4eaa8e42c205e0e6bec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:24.877049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c97b1945-376f-4c8a-8d0a-1d0219f2773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:25.034909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1036/2059] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:25.041745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:25.056539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595dddd-4dd7-4b11-b807-8949e1904437 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:25.061560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5595dddd-4dd7-4b11-b807-8949e1904437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:25.145973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595dddd-4dd7-4b11-b807-8949e1904437 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:25.149912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471917c5-aafb-4aaf-8fe9-fee0103eded9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:25.150676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471917c5-aafb-4aaf-8fe9-fee0103eded9 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:25.151185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471917c5-aafb-4aaf-8fe9-fee0103eded9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:25.151646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471917c5-aafb-4aaf-8fe9-fee0103eded9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:25.164046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471917c5-aafb-4aaf-8fe9-fee0103eded9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:25.166374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471917c5-aafb-4aaf-8fe9-fee0103eded9 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:25.181407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1024/2060] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:25 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:25.288485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:25.810115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b1945-376f-4c8a-8d0a-1d0219f2773d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:25.812308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c97b1945-376f-4c8a-8d0a-1d0219f2773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:25.832402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b1945-376f-4c8a-8d0a-1d0219f2773d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:25.838076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39e7d2-9d08-495c-bbf3-203078cdaea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:25.838818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39e7d2-9d08-495c-bbf3-203078cdaea1 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:25.839135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39e7d2-9d08-495c-bbf3-203078cdaea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:25.839630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39e7d2-9d08-495c-bbf3-203078cdaea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:25.912752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39e7d2-9d08-495c-bbf3-203078cdaea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:25.915050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39e7d2-9d08-495c-bbf3-203078cdaea1 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:25.928266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1037/2061] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:25 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:36.182457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=77 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:36.778373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=14 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:42.532594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f15b98-af3f-4cc3-adb0-f6fb4ee35b92 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:42.533178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f15b98-af3f-4cc3-adb0-f6fb4ee35b92 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:42.533178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f15b98-af3f-4cc3-adb0-f6fb4ee35b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:42.620607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-81f15b98-af3f-4cc3-adb0-f6fb4ee35b92 None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:42.625755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-81f15b98-af3f-4cc3-adb0-f6fb4ee35b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:42.726239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1025/2062] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:26:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:42.783673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39e7d2-9d08-495c-bbf3-203078cdaea1 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:42.785427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5e39e7d2-9d08-495c-bbf3-203078cdaea1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:42.914925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39e7d2-9d08-495c-bbf3-203078cdaea1 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:42.924288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115f41f-91db-407d-b6cc-c803851f5c42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:42.925785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115f41f-91db-407d-b6cc-c803851f5c42 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:42.926183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115f41f-91db-407d-b6cc-c803851f5c42 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:42.926595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115f41f-91db-407d-b6cc-c803851f5c42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:42.938375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115f41f-91db-407d-b6cc-c803851f5c42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:42.940121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115f41f-91db-407d-b6cc-c803851f5c42 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:42.953612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1038/2063] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:26:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:26:44.884679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f15b98-af3f-4cc3-adb0-f6fb4ee35b92 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:44.889039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-81f15b98-af3f-4cc3-adb0-f6fb4ee35b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:45.125705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f15b98-af3f-4cc3-adb0-f6fb4ee35b92 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:45.131080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:45.132508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:45.133041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:45.133558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] PATH_INFO: `/v3/users/5763586297b2472ea3ed013da067ad0b` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:45.135143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5763586297b2472ea3ed013da067ad0b)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:45.139518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:45.486987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5763586297b2472ea3ed013da067ad0b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:45.488134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5763586297b2472ea3ed013da067ad0b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:45.507984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5763586297b2472ea3ed013da067ad0b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:45.509228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5763586297b2472ea3ed013da067ad0b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:45.512939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:45.674619 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5763586297b2472ea3ed013da067ad0b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:45.688987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1026/2064] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:26:44 2022] DELETE /identity/v3/users/5763586297b2472ea3ed013da067ad0b => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:46.199315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115f41f-91db-407d-b6cc-c803851f5c42 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:46.233274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115f41f-91db-407d-b6cc-c803851f5c42 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:46.378127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:46.379323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:46.379778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:46.380228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] PATH_INFO: `/v3/projects/0e2640459def4ed3b1716c7cac92a6a7` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:46.382046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2640459def4ed3b1716c7cac92a6a7)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:46.384571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:46.585187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2640459def4ed3b1716c7cac92a6a7', 'request_id': 'req-3e902fcc-e92b-484a-afad-fc5bcd79001e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:46.597109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2640459def4ed3b1716c7cac92a6a7', 'request_id': 'req-3e902fcc-e92b-484a-afad-fc5bcd79001e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:46.712995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2640459def4ed3b1716c7cac92a6a7', 'request_id': 'req-3e902fcc-e92b-484a-afad-fc5bcd79001e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:46.726896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:46.734329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] The token cache is being invalidate because project 0e2640459def4ed3b1716c7cac92a6a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:46.734936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:46.735908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:46.738258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1039/2065] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:26:46 2022] DELETE /identity/v3/projects/0e2640459def4ed3b1716c7cac92a6a7 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:46.743615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:46.745838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:46.761209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f57cb7-7132-4753-b4bf-5ba79965e097 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:46.884291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:46.885833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:46.886309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:46.886830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] PATH_INFO: `/v3/users/f4ef72893291484f9d623680e1b54901` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:46.889914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ef72893291484f9d623680e1b54901)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:46.893497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:47.260690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ef72893291484f9d623680e1b54901'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:47.261915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ef72893291484f9d623680e1b54901'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:47.287039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ef72893291484f9d623680e1b54901'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:47.287943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ef72893291484f9d623680e1b54901'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:47.410125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e5e462b7-e3f5-46e8-bc62-4c63daaa099c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ef72893291484f9d623680e1b54901'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:47.433595 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1027/2066] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:26:46 2022] DELETE /identity/v3/users/f4ef72893291484f9d623680e1b54901 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:47.894089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:47.896356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:47.914435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902fcc-e92b-484a-afad-fc5bcd79001e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:48.041467 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:48.042386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:48.042773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:48.043044 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] PATH_INFO: `/v3/projects/e2d0451cc99743a9a3e600fe30b7b4be` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:48.046657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d0451cc99743a9a3e600fe30b7b4be)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:48.049982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:48.229897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d0451cc99743a9a3e600fe30b7b4be', 'request_id': 'req-7f7ff29f-39d7-482d-953b-adedb095ad56'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:48.255581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d0451cc99743a9a3e600fe30b7b4be', 'request_id': 'req-7f7ff29f-39d7-482d-953b-adedb095ad56'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:48.376403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d0451cc99743a9a3e600fe30b7b4be', 'request_id': 'req-7f7ff29f-39d7-482d-953b-adedb095ad56'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:48.408857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] The token cache is being invalidate because project e2d0451cc99743a9a3e600fe30b7b4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:48.409210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:48.413760 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7f7ff29f-39d7-482d-953b-adedb095ad56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:48.415871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1040/2067] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:26:47 2022] DELETE /identity/v3/projects/e2d0451cc99743a9a3e600fe30b7b4be => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:48.433156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6af201-69ac-4645-9596-bded4d601a1e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:48.434530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6af201-69ac-4645-9596-bded4d601a1e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:48.435491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6af201-69ac-4645-9596-bded4d601a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:48.473208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2e6af201-69ac-4645-9596-bded4d601a1e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:48.474633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2e6af201-69ac-4645-9596-bded4d601a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:48.580395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1028/2068] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:48.589174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccc202-b9d4-4611-bef2-f209ace01f59 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:48.589523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccc202-b9d4-4611-bef2-f209ace01f59 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:48.590127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbccc202-b9d4-4611-bef2-f209ace01f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:48.627754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-fbccc202-b9d4-4611-bef2-f209ace01f59 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:48.631329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fbccc202-b9d4-4611-bef2-f209ace01f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:48.646182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1041/2069] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:48.650488 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6af201-69ac-4645-9596-bded4d601a1e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:48.656108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:48.665837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6af201-69ac-4645-9596-bded4d601a1e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:48.675331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e4457-7682-482f-835e-c4adc082becd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:48.677028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e4457-7682-482f-835e-c4adc082becd admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:48.677683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e4457-7682-482f-835e-c4adc082becd admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:48.678072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e4457-7682-482f-835e-c4adc082becd admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:48.679637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e4457-7682-482f-835e-c4adc082becd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:26:48.680310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e4457-7682-482f-835e-c4adc082becd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:48.683639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e4457-7682-482f-835e-c4adc082becd admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:48.709963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1029/2070] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:26:48.711589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccc202-b9d4-4611-bef2-f209ace01f59 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:48.728775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbccc202-b9d4-4611-bef2-f209ace01f59 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:48.735981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04ee7a-8fe6-400a-9f1c-d5b271e73841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:48.736783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04ee7a-8fe6-400a-9f1c-d5b271e73841 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:48.737122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04ee7a-8fe6-400a-9f1c-d5b271e73841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:48.737395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04ee7a-8fe6-400a-9f1c-d5b271e73841 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:48.738109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04ee7a-8fe6-400a-9f1c-d5b271e73841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:48.740162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04ee7a-8fe6-400a-9f1c-d5b271e73841 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:48.852671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1042/2071] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:26:48 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:26:48.856936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e4457-7682-482f-835e-c4adc082becd admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:48.868515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e4457-7682-482f-835e-c4adc082becd admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:48.951830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0927d-324b-44e5-b79a-b78e2eb7199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:48.951830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0927d-324b-44e5-b79a-b78e2eb7199c admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:48.951830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0927d-324b-44e5-b79a-b78e2eb7199c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:48.951830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0927d-324b-44e5-b79a-b78e2eb7199c admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:48.952562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0927d-324b-44e5-b79a-b78e2eb7199c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:48.953863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0927d-324b-44e5-b79a-b78e2eb7199c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:49.096070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1030/2072] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:26:48 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:26:49.096902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04ee7a-8fe6-400a-9f1c-d5b271e73841 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:49.109162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04ee7a-8fe6-400a-9f1c-d5b271e73841 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:49.114317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aca97-d5c7-4877-8360-d7ac3449d014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:49.115200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aca97-d5c7-4877-8360-d7ac3449d014 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:49.115422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aca97-d5c7-4877-8360-d7ac3449d014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:49.115708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aca97-d5c7-4877-8360-d7ac3449d014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:49.116464 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058aca97-d5c7-4877-8360-d7ac3449d014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:49.118224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058aca97-d5c7-4877-8360-d7ac3449d014 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:49.132559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1043/2073] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:26:49.137008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0927d-324b-44e5-b79a-b78e2eb7199c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:49.150158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0927d-324b-44e5-b79a-b78e2eb7199c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:49.156456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180390e-e8cf-4ac5-9d32-edbc3592763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:49.157282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180390e-e8cf-4ac5-9d32-edbc3592763d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:49.157660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180390e-e8cf-4ac5-9d32-edbc3592763d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:49.158011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180390e-e8cf-4ac5-9d32-edbc3592763d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:49.158915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180390e-e8cf-4ac5-9d32-edbc3592763d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:49.160686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180390e-e8cf-4ac5-9d32-edbc3592763d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:49.176445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1031/2074] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:26:49.180471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aca97-d5c7-4877-8360-d7ac3449d014 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:49.199755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aca97-d5c7-4877-8360-d7ac3449d014 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:49.211920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cb5e1-456b-4773-b3da-53137f5d6b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:49.213030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cb5e1-456b-4773-b3da-53137f5d6b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:49.213313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cb5e1-456b-4773-b3da-53137f5d6b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:49.213608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cb5e1-456b-4773-b3da-53137f5d6b6b admin admin] PATH_INFO: `/v3/projects/3be6c3cdf062493184ab67ec3ec9940f/users/3cf7a5b535234632bb00a9b87ceb5354/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:49.242388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cb5e1-456b-4773-b3da-53137f5d6b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be6c3cdf062493184ab67ec3ec9940f, user_id=3cf7a5b535234632bb00a9b87ceb5354, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:49.248571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463cb5e1-456b-4773-b3da-53137f5d6b6b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:49.346016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1044/2075] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:26:49 2022] PUT /identity/v3/projects/3be6c3cdf062493184ab67ec3ec9940f/users/3cf7a5b535234632bb00a9b87ceb5354/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:49.348653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f2e03-f4b1-4496-83fa-ae7d232ac8cd None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:49.349131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f2e03-f4b1-4496-83fa-ae7d232ac8cd None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:49.349757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f2e03-f4b1-4496-83fa-ae7d232ac8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:49.383288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-696f2e03-f4b1-4496-83fa-ae7d232ac8cd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:49.385250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-696f2e03-f4b1-4496-83fa-ae7d232ac8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:49.508717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1032/2076] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:49.521925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cb5e1-456b-4773-b3da-53137f5d6b6b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:49.524543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-463cb5e1-456b-4773-b3da-53137f5d6b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:49.561614 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:49.604591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463cb5e1-456b-4773-b3da-53137f5d6b6b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:49.612929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5d964-fe42-4d3c-9fdd-5e31190afc55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:49.614086 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5d964-fe42-4d3c-9fdd-5e31190afc55 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:49.614628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5d964-fe42-4d3c-9fdd-5e31190afc55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:49.615126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5d964-fe42-4d3c-9fdd-5e31190afc55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:49.634355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5d964-fe42-4d3c-9fdd-5e31190afc55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:49.636985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5d964-fe42-4d3c-9fdd-5e31190afc55 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:49.659599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1045/2077] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:26:56.536311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f2e03-f4b1-4496-83fa-ae7d232ac8cd None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:56.543042 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-696f2e03-f4b1-4496-83fa-ae7d232ac8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:56.613262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f2e03-f4b1-4496-83fa-ae7d232ac8cd None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:56.654610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:56.682721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:56.683310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:56.684124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] PATH_INFO: `/v3/users/fc642bf43bb24b7b8c10b2d675b0fea5` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:56.697820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc642bf43bb24b7b8c10b2d675b0fea5)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:56.701934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:57.171954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc642bf43bb24b7b8c10b2d675b0fea5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:57.173339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc642bf43bb24b7b8c10b2d675b0fea5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:57.209052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:26:57.218981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc642bf43bb24b7b8c10b2d675b0fea5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:57.222114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc642bf43bb24b7b8c10b2d675b0fea5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:57.350801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc642bf43bb24b7b8c10b2d675b0fea5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:57.415101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1033/2078] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:26:56 2022] DELETE /identity/v3/users/fc642bf43bb24b7b8c10b2d675b0fea5 => generated 0 bytes in 880 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:57.423217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e558b-9302-46a8-8155-c9e0bd3e9177 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:57.424123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e558b-9302-46a8-8155-c9e0bd3e9177 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:57.424123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e558b-9302-46a8-8155-c9e0bd3e9177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:57.462813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ab9e558b-9302-46a8-8155-c9e0bd3e9177 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:57.471906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ab9e558b-9302-46a8-8155-c9e0bd3e9177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:57.581481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1046/2079] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:26:57.594759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:57.597185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:57.848417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d469c29-c805-4445-9235-39bb0d0acdc4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:57.856954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b72d57-fe8c-4de2-bc14-c6360273f330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:57.858386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b72d57-fe8c-4de2-bc14-c6360273f330 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:57.858901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b72d57-fe8c-4de2-bc14-c6360273f330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:57.859310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b72d57-fe8c-4de2-bc14-c6360273f330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:57.898564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b72d57-fe8c-4de2-bc14-c6360273f330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:57.903588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b72d57-fe8c-4de2-bc14-c6360273f330 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:57.944010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1034/2080] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:26:58.813606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e558b-9302-46a8-8155-c9e0bd3e9177 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:58.815557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ab9e558b-9302-46a8-8155-c9e0bd3e9177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:58.834991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e558b-9302-46a8-8155-c9e0bd3e9177 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:58.846477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:58.849999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:58.850293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:58.850699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] PATH_INFO: `/v3/projects/6b4cbc799d504116a0666915febb872c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:58.860046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b4cbc799d504116a0666915febb872c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:58.870052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:59.068679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b4cbc799d504116a0666915febb872c', 'request_id': 'req-bd2cf1af-7c39-4225-b319-fd88a502d81b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.081415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b4cbc799d504116a0666915febb872c', 'request_id': 'req-bd2cf1af-7c39-4225-b319-fd88a502d81b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.207389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b4cbc799d504116a0666915febb872c', 'request_id': 'req-bd2cf1af-7c39-4225-b319-fd88a502d81b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.230077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] The token cache is being invalidate because project 6b4cbc799d504116a0666915febb872c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:26:59.230512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.231309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-bd2cf1af-7c39-4225-b319-fd88a502d81b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.233221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1047/2081] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:26:58 2022] DELETE /identity/v3/projects/6b4cbc799d504116a0666915febb872c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:59.237604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b72d57-fe8c-4de2-bc14-c6360273f330 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:26:59.251749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b72d57-fe8c-4de2-bc14-c6360273f330 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:26:59.530395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:26:59.533198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:59.535165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:59.536403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] PATH_INFO: `/v3/users/cbb5e189a15d4eaa8e42c205e0e6bec5` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:59.539766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb5e189a15d4eaa8e42c205e0e6bec5)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:26:59.547723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:26:59.792790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb5e189a15d4eaa8e42c205e0e6bec5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.793607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb5e189a15d4eaa8e42c205e0e6bec5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.826281 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dbaf9-36f3-4282-a23b-e478606528e6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:26:59.827460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dbaf9-36f3-4282-a23b-e478606528e6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:26:59.827954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dbaf9-36f3-4282-a23b-e478606528e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:26:59.834877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb5e189a15d4eaa8e42c205e0e6bec5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.838674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb5e189a15d4eaa8e42c205e0e6bec5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.881042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-692dbaf9-36f3-4282-a23b-e478606528e6 None None] MFA Rules not processed for user `3cf7a5b535234632bb00a9b87ceb5354`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:26:59.883639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-692dbaf9-36f3-4282-a23b-e478606528e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:26:59.922857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb5e189a15d4eaa8e42c205e0e6bec5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:26:59.951892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1035/2082] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:26:59 2022] DELETE /identity/v3/users/cbb5e189a15d4eaa8e42c205e0e6bec5 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:26:59.990154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:00.035696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1048/2083] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:26:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:00.041076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:00.044604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:00.053787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:00.179290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37620e-f458-47f6-843f-511761c10dcb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:00.188748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75a30f-232f-4d52-a97a-7f47725334c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:00.190150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75a30f-232f-4d52-a97a-7f47725334c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:00.190703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75a30f-232f-4d52-a97a-7f47725334c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:00.191199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75a30f-232f-4d52-a97a-7f47725334c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:00.192597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75a30f-232f-4d52-a97a-7f47725334c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:00.196602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75a30f-232f-4d52-a97a-7f47725334c8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:00.440473 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1036/2084] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:00 2022] POST /identity/v3/projects => generated 385 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:00.445639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692dbaf9-36f3-4282-a23b-e478606528e6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:00.494008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692dbaf9-36f3-4282-a23b-e478606528e6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:00.629061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d61a-096b-4e01-a19a-bbebd4977c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:00.629985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d61a-096b-4e01-a19a-bbebd4977c45 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:00.630429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d61a-096b-4e01-a19a-bbebd4977c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:00.630852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6d61a-096b-4e01-a19a-bbebd4977c45 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:00.632037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6d61a-096b-4e01-a19a-bbebd4977c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:00.634976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6d61a-096b-4e01-a19a-bbebd4977c45 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:01.028528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75a30f-232f-4d52-a97a-7f47725334c8 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:01.039600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1e75a30f-232f-4d52-a97a-7f47725334c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:01.057644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1049/2085] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:00 2022] POST /identity/v3/users => generated 299 bytes in 613 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:01.059936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d61a-096b-4e01-a19a-bbebd4977c45 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:01.072835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75a30f-232f-4d52-a97a-7f47725334c8 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:01.076988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6d61a-096b-4e01-a19a-bbebd4977c45 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:01.082275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:01.083277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:01.083572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:01.083854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] PATH_INFO: `/v3/projects/a8818307692d498db2541fb2a6dcc6d1` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:01.085279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8818307692d498db2541fb2a6dcc6d1)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:01.087770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:01.100499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fd7cc-6acc-4774-aa20-27f35528374a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:01.101498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fd7cc-6acc-4774-aa20-27f35528374a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:01.102062 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fd7cc-6acc-4774-aa20-27f35528374a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:01.102657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fd7cc-6acc-4774-aa20-27f35528374a admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:01.103787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fd7cc-6acc-4774-aa20-27f35528374a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:01.105989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fd7cc-6acc-4774-aa20-27f35528374a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:01.124872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1050/2086] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:01.130365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fd7cc-6acc-4774-aa20-27f35528374a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:01.161106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fd7cc-6acc-4774-aa20-27f35528374a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:01.177495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed684c-bf55-4b6b-a4bf-085d4c9fdc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:01.178807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed684c-bf55-4b6b-a4bf-085d4c9fdc93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:01.179202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed684c-bf55-4b6b-a4bf-085d4c9fdc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:01.182193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed684c-bf55-4b6b-a4bf-085d4c9fdc93 admin admin] PATH_INFO: `/v3/projects/0cf5d50226094a7a8d8e2e968d8a0cb1/users/1e57847c26ee48108c7285a91a488218/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:01.223774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed684c-bf55-4b6b-a4bf-085d4c9fdc93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf5d50226094a7a8d8e2e968d8a0cb1, user_id=1e57847c26ee48108c7285a91a488218, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:01.228279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed684c-bf55-4b6b-a4bf-085d4c9fdc93 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:01.453112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1051/2087] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:01 2022] PUT /identity/v3/projects/0cf5d50226094a7a8d8e2e968d8a0cb1/users/1e57847c26ee48108c7285a91a488218/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:01.454474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed684c-bf55-4b6b-a4bf-085d4c9fdc93 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:01.475937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8818307692d498db2541fb2a6dcc6d1', 'request_id': 'req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:01.488833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8818307692d498db2541fb2a6dcc6d1', 'request_id': 'req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:01.591006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8818307692d498db2541fb2a6dcc6d1', 'request_id': 'req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:01.599964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed684c-bf55-4b6b-a4bf-085d4c9fdc93 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:01.613127 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] The token cache is being invalidate because project a8818307692d498db2541fb2a6dcc6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:01.613426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:01.616147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-da75039e-2e9e-4402-8c2c-c08ca2ec5dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:01.625263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1037/2088] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:01 2022] DELETE /identity/v3/projects/a8818307692d498db2541fb2a6dcc6d1 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:01.630317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82a1de-9410-4df7-bbff-887db914315a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:01.630624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82a1de-9410-4df7-bbff-887db914315a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:01.632825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82a1de-9410-4df7-bbff-887db914315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:01.676602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ae82a1de-9410-4df7-bbff-887db914315a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:01.677910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ae82a1de-9410-4df7-bbff-887db914315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:01.691067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:01.708527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04788e1b-83a2-4358-8b62-fbdd2cb8e1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:01.709150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788e1b-83a2-4358-8b62-fbdd2cb8e1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:01.710702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788e1b-83a2-4358-8b62-fbdd2cb8e1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:01.710702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04788e1b-83a2-4358-8b62-fbdd2cb8e1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:01.710702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04788e1b-83a2-4358-8b62-fbdd2cb8e1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:01.712532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04788e1b-83a2-4358-8b62-fbdd2cb8e1c1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:01.754176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1052/2089] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:01.760093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04788e1b-83a2-4358-8b62-fbdd2cb8e1c1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:01.762388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-04788e1b-83a2-4358-8b62-fbdd2cb8e1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:01.783163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1038/2090] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:01.783163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e18490-b5f4-4582-93f3-f0377a5d2e8b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:01.783630 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e18490-b5f4-4582-93f3-f0377a5d2e8b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:01.785800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e18490-b5f4-4582-93f3-f0377a5d2e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:01.800304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04788e1b-83a2-4358-8b62-fbdd2cb8e1c1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:01.807372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2e009-75de-49fb-95df-7124d8dc48b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:01.808410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2e009-75de-49fb-95df-7124d8dc48b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:01.811672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2e009-75de-49fb-95df-7124d8dc48b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:01.811672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2e009-75de-49fb-95df-7124d8dc48b5 admin admin] PATH_INFO: `/v3/projects/0cf5d50226094a7a8d8e2e968d8a0cb1/users/1e57847c26ee48108c7285a91a488218/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:01.816506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2e009-75de-49fb-95df-7124d8dc48b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf5d50226094a7a8d8e2e968d8a0cb1, user_id=1e57847c26ee48108c7285a91a488218, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:01.821504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2e009-75de-49fb-95df-7124d8dc48b5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:01.825698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-84e18490-b5f4-4582-93f3-f0377a5d2e8b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:01.827125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-84e18490-b5f4-4582-93f3-f0377a5d2e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:01.837377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1039/2091] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:01.840997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e18490-b5f4-4582-93f3-f0377a5d2e8b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:01.847873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:01.849490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1053/2092] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:01 2022] PUT /identity/v3/projects/0cf5d50226094a7a8d8e2e968d8a0cb1/users/1e57847c26ee48108c7285a91a488218/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:01.856839 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e18490-b5f4-4582-93f3-f0377a5d2e8b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:01.936734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5bcc2-a14e-41da-a8ab-aaa9bed5d46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:01.937667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5bcc2-a14e-41da-a8ab-aaa9bed5d46a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:01.937949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5bcc2-a14e-41da-a8ab-aaa9bed5d46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:01.938324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5bcc2-a14e-41da-a8ab-aaa9bed5d46a admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:01.939220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5bcc2-a14e-41da-a8ab-aaa9bed5d46a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:27:01.939668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5bcc2-a14e-41da-a8ab-aaa9bed5d46a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:01.942056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e5bcc2-a14e-41da-a8ab-aaa9bed5d46a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:01.961015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1040/2093] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:27:01.962977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2e009-75de-49fb-95df-7124d8dc48b5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:01.975778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2e009-75de-49fb-95df-7124d8dc48b5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:01.981697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973ca79-6ebf-41d0-a6bb-36667afaa517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:01.988081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973ca79-6ebf-41d0-a6bb-36667afaa517 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:01.988081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973ca79-6ebf-41d0-a6bb-36667afaa517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:01.988081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973ca79-6ebf-41d0-a6bb-36667afaa517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:01.988081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973ca79-6ebf-41d0-a6bb-36667afaa517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:01.988081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973ca79-6ebf-41d0-a6bb-36667afaa517 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:02.214145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1054/2094] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:01 2022] POST /identity/v3/projects => generated 363 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:27:02.216777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5bcc2-a14e-41da-a8ab-aaa9bed5d46a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:02.235431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5bcc2-a14e-41da-a8ab-aaa9bed5d46a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:02.426373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de802c7-5433-49f2-8c71-457184dfbf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:02.427984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de802c7-5433-49f2-8c71-457184dfbf8d admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:02.428276 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de802c7-5433-49f2-8c71-457184dfbf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:02.428636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de802c7-5433-49f2-8c71-457184dfbf8d admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:02.429359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de802c7-5433-49f2-8c71-457184dfbf8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:02.432330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de802c7-5433-49f2-8c71-457184dfbf8d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:02.597065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1041/2095] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:02 2022] POST /identity/v3/users => generated 282 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:02.601436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973ca79-6ebf-41d0-a6bb-36667afaa517 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:02.621107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973ca79-6ebf-41d0-a6bb-36667afaa517 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:02.640922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fbe26-e6e8-4bbc-bdd7-b1a97c86e1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:02.645854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fbe26-e6e8-4bbc-bdd7-b1a97c86e1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:02.645854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fbe26-e6e8-4bbc-bdd7-b1a97c86e1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:02.645854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302fbe26-e6e8-4bbc-bdd7-b1a97c86e1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:02.645854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fbe26-e6e8-4bbc-bdd7-b1a97c86e1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:02.648611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302fbe26-e6e8-4bbc-bdd7-b1a97c86e1d6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:02.678500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1055/2096] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:27:02.680021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de802c7-5433-49f2-8c71-457184dfbf8d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:02.698174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de802c7-5433-49f2-8c71-457184dfbf8d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:02.710053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21369b9-f2c8-4f05-b280-0bc24f8b7a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:02.710843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21369b9-f2c8-4f05-b280-0bc24f8b7a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:02.711134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21369b9-f2c8-4f05-b280-0bc24f8b7a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:02.711424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21369b9-f2c8-4f05-b280-0bc24f8b7a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:02.712220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21369b9-f2c8-4f05-b280-0bc24f8b7a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:02.713811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21369b9-f2c8-4f05-b280-0bc24f8b7a8b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:02.735391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1042/2097] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:02.736311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fbe26-e6e8-4bbc-bdd7-b1a97c86e1d6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:02.751686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302fbe26-e6e8-4bbc-bdd7-b1a97c86e1d6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:02.758667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b4681-a0f0-45cc-98a1-da212f344c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:02.758917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b4681-a0f0-45cc-98a1-da212f344c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:02.759279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b4681-a0f0-45cc-98a1-da212f344c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:02.759585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b4681-a0f0-45cc-98a1-da212f344c42 admin admin] PATH_INFO: `/v3/projects/133e12dca2544f9b910713b2cc5292e5/users/c2ff31f478a54a85a81d41b5296914b7/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:02.785792 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b4681-a0f0-45cc-98a1-da212f344c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=133e12dca2544f9b910713b2cc5292e5, user_id=c2ff31f478a54a85a81d41b5296914b7, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:02.792516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b4681-a0f0-45cc-98a1-da212f344c42 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:02.912929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1056/2098] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:02 2022] PUT /identity/v3/projects/133e12dca2544f9b910713b2cc5292e5/users/c2ff31f478a54a85a81d41b5296914b7/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:02.924685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf5e70-a384-422e-b6d1-083152872d2c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:02.925284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf5e70-a384-422e-b6d1-083152872d2c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:02.925720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf5e70-a384-422e-b6d1-083152872d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:03.014962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9bcf5e70-a384-422e-b6d1-083152872d2c None None] MFA Rules not processed for user `c2ff31f478a54a85a81d41b5296914b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:03.016507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9bcf5e70-a384-422e-b6d1-083152872d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:03.147335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1043/2099] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:03.156380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b4681-a0f0-45cc-98a1-da212f344c42 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:03.333812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b4681-a0f0-45cc-98a1-da212f344c42 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:03.341754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68500b6f-7d10-4fd5-989a-6a6ac243a846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:03.343432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68500b6f-7d10-4fd5-989a-6a6ac243a846 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:03.343812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68500b6f-7d10-4fd5-989a-6a6ac243a846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:03.344102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68500b6f-7d10-4fd5-989a-6a6ac243a846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:03.344911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68500b6f-7d10-4fd5-989a-6a6ac243a846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:03.348102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68500b6f-7d10-4fd5-989a-6a6ac243a846 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:03.504867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1057/2100] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:03 2022] POST /identity/v3/projects => generated 365 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:27:03.508683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf5e70-a384-422e-b6d1-083152872d2c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:03.527643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf5e70-a384-422e-b6d1-083152872d2c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:03.534328 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:03.726100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ae37b-fb39-4516-a42f-3d461dd99929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:03.727734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ae37b-fb39-4516-a42f-3d461dd99929 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:03.727886 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ae37b-fb39-4516-a42f-3d461dd99929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:03.728140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ae37b-fb39-4516-a42f-3d461dd99929 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:03.732425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ae37b-fb39-4516-a42f-3d461dd99929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:03.743391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ae37b-fb39-4516-a42f-3d461dd99929 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:03.914953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1044/2101] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:03 2022] POST /identity/v3/users => generated 289 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:03.918036 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68500b6f-7d10-4fd5-989a-6a6ac243a846 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:03.948118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68500b6f-7d10-4fd5-989a-6a6ac243a846 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:03.961499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18ea62-fe22-4fe9-a5e5-d0f17112392b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:03.962402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18ea62-fe22-4fe9-a5e5-d0f17112392b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:03.962878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18ea62-fe22-4fe9-a5e5-d0f17112392b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:03.963164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18ea62-fe22-4fe9-a5e5-d0f17112392b admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:03.964132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18ea62-fe22-4fe9-a5e5-d0f17112392b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:03.967013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18ea62-fe22-4fe9-a5e5-d0f17112392b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:03.983780 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1058/2102] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:03.989912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ae37b-fb39-4516-a42f-3d461dd99929 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:04.004410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ae37b-fb39-4516-a42f-3d461dd99929 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:04.029785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c95fc9-8d8c-4efe-9195-aca5e7cae422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:04.038555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c95fc9-8d8c-4efe-9195-aca5e7cae422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:04.039123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c95fc9-8d8c-4efe-9195-aca5e7cae422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:04.040471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c95fc9-8d8c-4efe-9195-aca5e7cae422 admin admin] PATH_INFO: `/v3/projects/ebb1e6176bf0438291c4b90b34ed7a24/users/d0c292f2b5864afd8f80692a54a2b61a/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:04.066397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c95fc9-8d8c-4efe-9195-aca5e7cae422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb1e6176bf0438291c4b90b34ed7a24, user_id=d0c292f2b5864afd8f80692a54a2b61a, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:04.069661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c95fc9-8d8c-4efe-9195-aca5e7cae422 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:04.220113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1045/2103] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:03 2022] PUT /identity/v3/projects/ebb1e6176bf0438291c4b90b34ed7a24/users/d0c292f2b5864afd8f80692a54a2b61a/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:04.221425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18ea62-fe22-4fe9-a5e5-d0f17112392b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:04.238631 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18ea62-fe22-4fe9-a5e5-d0f17112392b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:04.380325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:04.423707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f89075-d319-4c6a-9a91-dd8cdd2c6919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:04.424380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f89075-d319-4c6a-9a91-dd8cdd2c6919 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:04.424758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f89075-d319-4c6a-9a91-dd8cdd2c6919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:04.425686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f89075-d319-4c6a-9a91-dd8cdd2c6919 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:04.427015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f89075-d319-4c6a-9a91-dd8cdd2c6919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:04.428915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f89075-d319-4c6a-9a91-dd8cdd2c6919 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:04.449229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1059/2104] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:04.450142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c95fc9-8d8c-4efe-9195-aca5e7cae422 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:04.470844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c95fc9-8d8c-4efe-9195-aca5e7cae422 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:04.489766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b52063-2d04-44b8-9a10-2cc97ae33a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:04.489766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52063-2d04-44b8-9a10-2cc97ae33a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:04.489766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52063-2d04-44b8-9a10-2cc97ae33a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:04.489766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b52063-2d04-44b8-9a10-2cc97ae33a1e admin admin] PATH_INFO: `/v3/projects/ebb1e6176bf0438291c4b90b34ed7a24/users/d0c292f2b5864afd8f80692a54a2b61a/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:04.494674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b52063-2d04-44b8-9a10-2cc97ae33a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb1e6176bf0438291c4b90b34ed7a24, user_id=d0c292f2b5864afd8f80692a54a2b61a, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:04.502997 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b52063-2d04-44b8-9a10-2cc97ae33a1e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:04.531506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1046/2105] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:04 2022] PUT /identity/v3/projects/ebb1e6176bf0438291c4b90b34ed7a24/users/d0c292f2b5864afd8f80692a54a2b61a/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:04.541913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272591-b7f8-451a-a5ee-5dbc6bc8f3df None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:04.545051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272591-b7f8-451a-a5ee-5dbc6bc8f3df None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:04.547573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272591-b7f8-451a-a5ee-5dbc6bc8f3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:04.669385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f2272591-b7f8-451a-a5ee-5dbc6bc8f3df None None] MFA Rules not processed for user `d0c292f2b5864afd8f80692a54a2b61a`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:04.671066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f2272591-b7f8-451a-a5ee-5dbc6bc8f3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:04.844406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1060/2106] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:04.858189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b52063-2d04-44b8-9a10-2cc97ae33a1e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:04.861848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-56b52063-2d04-44b8-9a10-2cc97ae33a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:04.976979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b52063-2d04-44b8-9a10-2cc97ae33a1e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:04.982770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664699ff-03a8-4f6f-b2fb-3bc29cbadd72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:04.983716 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664699ff-03a8-4f6f-b2fb-3bc29cbadd72 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:04.986451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664699ff-03a8-4f6f-b2fb-3bc29cbadd72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:04.989717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664699ff-03a8-4f6f-b2fb-3bc29cbadd72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:05.000803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664699ff-03a8-4f6f-b2fb-3bc29cbadd72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:05.002777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664699ff-03a8-4f6f-b2fb-3bc29cbadd72 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:05.022699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1047/2107] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:05.320228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2272591-b7f8-451a-a5ee-5dbc6bc8f3df None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:05.333052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2272591-b7f8-451a-a5ee-5dbc6bc8f3df None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:05.348885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78513e9c-731c-4354-8133-f58c9a7bbf70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:05.354904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78513e9c-731c-4354-8133-f58c9a7bbf70 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:05.354904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78513e9c-731c-4354-8133-f58c9a7bbf70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:05.354904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78513e9c-731c-4354-8133-f58c9a7bbf70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:05.626056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78513e9c-731c-4354-8133-f58c9a7bbf70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:05.630000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78513e9c-731c-4354-8133-f58c9a7bbf70 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:05.661942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1061/2108] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:05 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:27:09.424758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959474b0-4a3c-4544-9e9c-318652db040b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:09.426358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959474b0-4a3c-4544-9e9c-318652db040b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:09.427157 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959474b0-4a3c-4544-9e9c-318652db040b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:09.456123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-959474b0-4a3c-4544-9e9c-318652db040b None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:09.459229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-959474b0-4a3c-4544-9e9c-318652db040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:09.577410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1048/2109] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 19 12:27:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:09.594014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78513e9c-731c-4354-8133-f58c9a7bbf70 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:09.596054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-78513e9c-731c-4354-8133-f58c9a7bbf70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:09.790013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78513e9c-731c-4354-8133-f58c9a7bbf70 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:09.812757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10e487-9913-450e-824d-c982e0affc3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:09.814214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10e487-9913-450e-824d-c982e0affc3f service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:09.814736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10e487-9913-450e-824d-c982e0affc3f service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:09.815031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10e487-9913-450e-824d-c982e0affc3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:09.830714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:09.837503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10e487-9913-450e-824d-c982e0affc3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:09.842572 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10e487-9913-450e-824d-c982e0affc3f service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:09.857602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1062/2110] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:27:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:27:10.299794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959474b0-4a3c-4544-9e9c-318652db040b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:10.322900 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:10.488138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959474b0-4a3c-4544-9e9c-318652db040b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:10.494946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:10.495870 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:10.496161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:10.497141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] PATH_INFO: `/v3/users/c2ff31f478a54a85a81d41b5296914b7` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:10.498555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ff31f478a54a85a81d41b5296914b7)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:10.501134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:10.656029 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf91487-3fdc-468b-aea6-c69f7c516c06 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:10.656875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf91487-3fdc-468b-aea6-c69f7c516c06 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:10.656875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf91487-3fdc-468b-aea6-c69f7c516c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:10.692345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8cf91487-3fdc-468b-aea6-c69f7c516c06 None None] MFA Rules not processed for user `5c35bd3640694b0dbce13c713fd68306`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:10.693611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8cf91487-3fdc-468b-aea6-c69f7c516c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:10.812016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1063/2111] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:10.860469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ff31f478a54a85a81d41b5296914b7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:10.861818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ff31f478a54a85a81d41b5296914b7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:10.883502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ff31f478a54a85a81d41b5296914b7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:10.884838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ff31f478a54a85a81d41b5296914b7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:11.064536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ceda1a74-8c0d-4618-9564-456c88b4c6e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ff31f478a54a85a81d41b5296914b7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:11.117550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1049/2112] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:10 2022] DELETE /identity/v3/users/c2ff31f478a54a85a81d41b5296914b7 => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:11.127163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15227b-f2a1-4333-87ad-bcfb166620e5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:11.127439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15227b-f2a1-4333-87ad-bcfb166620e5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:11.127721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15227b-f2a1-4333-87ad-bcfb166620e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:11.180740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cf15227b-f2a1-4333-87ad-bcfb166620e5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:11.182062 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cf15227b-f2a1-4333-87ad-bcfb166620e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:11.233502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a211-eb15-45ed-8071-c4200791e378 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:11.234539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a211-eb15-45ed-8071-c4200791e378 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:11.236718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a211-eb15-45ed-8071-c4200791e378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:11.344277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1064/2113] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:11.350851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-88c4a211-eb15-45ed-8071-c4200791e378 None None] MFA Rules not processed for user `1e57847c26ee48108c7285a91a488218`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:11.353086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-88c4a211-eb15-45ed-8071-c4200791e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:11.362328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15227b-f2a1-4333-87ad-bcfb166620e5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:11.362328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cf15227b-f2a1-4333-87ad-bcfb166620e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:11.474812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1050/2114] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:27:11.489094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4a211-eb15-45ed-8071-c4200791e378 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:11.494223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-88c4a211-eb15-45ed-8071-c4200791e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:11.495505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15227b-f2a1-4333-87ad-bcfb166620e5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:11.504107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7ffcf-089c-4808-b8d7-5f2d1fc9f4c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:11.504783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7ffcf-089c-4808-b8d7-5f2d1fc9f4c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:11.505237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7ffcf-089c-4808-b8d7-5f2d1fc9f4c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:11.505711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7ffcf-089c-4808-b8d7-5f2d1fc9f4c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:11.511082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:11.518738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7ffcf-089c-4808-b8d7-5f2d1fc9f4c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:11.520466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7ffcf-089c-4808-b8d7-5f2d1fc9f4c6 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:11.533188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1065/2115] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:27:11.630540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4a211-eb15-45ed-8071-c4200791e378 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:11.639844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301cca99-23ac-4c88-a6ed-37a8c8a7838b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9969632e796345fa9671a45cf61ede3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:11.640898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cca99-23ac-4c88-a6ed-37a8c8a7838b service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:11.641380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cca99-23ac-4c88-a6ed-37a8c8a7838b service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:11.641846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301cca99-23ac-4c88-a6ed-37a8c8a7838b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:11.644289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-301cca99-23ac-4c88-a6ed-37a8c8a7838b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:11.756256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cca99-23ac-4c88-a6ed-37a8c8a7838b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:11.757749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301cca99-23ac-4c88-a6ed-37a8c8a7838b service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:11.786929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1051/2116] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:12.261929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7ffcf-089c-4808-b8d7-5f2d1fc9f4c6 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:12.265879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6bb7ffcf-089c-4808-b8d7-5f2d1fc9f4c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:12.309588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7ffcf-089c-4808-b8d7-5f2d1fc9f4c6 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:12.317433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:12.319377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:12.320128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:12.320739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] PATH_INFO: `/v3/projects/133e12dca2544f9b910713b2cc5292e5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:12.323033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=133e12dca2544f9b910713b2cc5292e5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:12.327342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:12.404552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081a4cd-02c8-4074-9f72-3b09fc6318d3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:12.404850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081a4cd-02c8-4074-9f72-3b09fc6318d3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:12.405147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081a4cd-02c8-4074-9f72-3b09fc6318d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:12.427333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0081a4cd-02c8-4074-9f72-3b09fc6318d3 None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:12.429279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0081a4cd-02c8-4074-9f72-3b09fc6318d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:12.512879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:12.517089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1052/2117] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:27:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:27:12.544417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081a4cd-02c8-4074-9f72-3b09fc6318d3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:12.547351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0081a4cd-02c8-4074-9f72-3b09fc6318d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:12.555856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '133e12dca2544f9b910713b2cc5292e5', 'request_id': 'req-b7327c62-f0dd-4955-bc8b-c2300d5376d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:12.568229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '133e12dca2544f9b910713b2cc5292e5', 'request_id': 'req-b7327c62-f0dd-4955-bc8b-c2300d5376d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:12.632744 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:12.652307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081a4cd-02c8-4074-9f72-3b09fc6318d3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:12.723584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e9077-377c-4d9d-af84-9aca6effb6f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:12.724247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e9077-377c-4d9d-af84-9aca6effb6f9 service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:12.724531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e9077-377c-4d9d-af84-9aca6effb6f9 service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:12.724809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e9077-377c-4d9d-af84-9aca6effb6f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:12.736039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '133e12dca2544f9b910713b2cc5292e5', 'request_id': 'req-b7327c62-f0dd-4955-bc8b-c2300d5376d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:12.756048 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] The token cache is being invalidate because project 133e12dca2544f9b910713b2cc5292e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:12.756355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:12.757052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:12.760223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1066/2118] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:12 2022] DELETE /identity/v3/projects/133e12dca2544f9b910713b2cc5292e5 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:12.765141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:12.766826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:12.814979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e9077-377c-4d9d-af84-9aca6effb6f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:12.816804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e9077-377c-4d9d-af84-9aca6effb6f9 service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:12.827195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1053/2119] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:27:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:27:12.854874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7327c62-f0dd-4955-bc8b-c2300d5376d6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:12.860858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:12.861769 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:12.862178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:12.862549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] PATH_INFO: `/v3/users/d0c292f2b5864afd8f80692a54a2b61a` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:12.863838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c292f2b5864afd8f80692a54a2b61a)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:12.865754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:13.242952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c292f2b5864afd8f80692a54a2b61a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:13.258643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c292f2b5864afd8f80692a54a2b61a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:13.259708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c292f2b5864afd8f80692a54a2b61a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:13.260656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c292f2b5864afd8f80692a54a2b61a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:13.385708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c292f2b5864afd8f80692a54a2b61a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:13.432732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1067/2120] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:12 2022] DELETE /identity/v3/users/d0c292f2b5864afd8f80692a54a2b61a => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:13.966479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e9077-377c-4d9d-af84-9aca6effb6f9 service swift] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:13.968972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0a9e9077-377c-4d9d-af84-9aca6effb6f9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:14.164005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e9077-377c-4d9d-af84-9aca6effb6f9 service swift] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:14.186545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:14.210118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:14.211738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:14.281035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bd58a-8c12-4654-bd6a-f3bd35bdb7cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:14.281986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bd58a-8c12-4654-bd6a-f3bd35bdb7cd service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:14.282554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bd58a-8c12-4654-bd6a-f3bd35bdb7cd service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:14.283000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bd58a-8c12-4654-bd6a-f3bd35bdb7cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:14.287269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-433bd58a-8c12-4654-bd6a-f3bd35bdb7cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:14.326834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bd58a-8c12-4654-bd6a-f3bd35bdb7cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:14.328546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bd58a-8c12-4654-bd6a-f3bd35bdb7cd service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:14.359142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:14.381075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1054/2121] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:14.406563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d834e-b786-4c66-b660-512f23f59983 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:14.424421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:14.433755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:14.434513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:14.436253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] PATH_INFO: `/v3/projects/ebb1e6176bf0438291c4b90b34ed7a24` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:14.439811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb1e6176bf0438291c4b90b34ed7a24)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:14.442424 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:14.617920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb1e6176bf0438291c4b90b34ed7a24', 'request_id': 'req-7569f6d5-6e9b-45ce-898a-130fc62def29'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:14.628519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb1e6176bf0438291c4b90b34ed7a24', 'request_id': 'req-7569f6d5-6e9b-45ce-898a-130fc62def29'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:15.150144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb1e6176bf0438291c4b90b34ed7a24', 'request_id': 'req-7569f6d5-6e9b-45ce-898a-130fc62def29'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:15.170470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] The token cache is being invalidate because project ebb1e6176bf0438291c4b90b34ed7a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:15.170856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:15.171564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:15.173560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1068/2122] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:14 2022] DELETE /identity/v3/projects/ebb1e6176bf0438291c4b90b34ed7a24 => generated 0 bytes in 965 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:15.179957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37026a-bc41-4525-bcaf-b67563ea42c4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:15.181289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37026a-bc41-4525-bcaf-b67563ea42c4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:15.181289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37026a-bc41-4525-bcaf-b67563ea42c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:15.246895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9d37026a-bc41-4525-bcaf-b67563ea42c4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:15.252323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9d37026a-bc41-4525-bcaf-b67563ea42c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:15.254337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:15.256114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:15.349990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569f6d5-6e9b-45ce-898a-130fc62def29 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:15.356501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:15.357876 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:15.358385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:15.359783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] PATH_INFO: `/v3/users/7caa126f7eab4eef8db03374e82a6ab9` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:15.361726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7caa126f7eab4eef8db03374e82a6ab9)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:15.366333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:15.446592 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1055/2123] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:27:15.455020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3e157-f12e-4a65-a3fd-a8962491fa94 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:15.455434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3e157-f12e-4a65-a3fd-a8962491fa94 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:15.455773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3e157-f12e-4a65-a3fd-a8962491fa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:15.492297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-33b3e157-f12e-4a65-a3fd-a8962491fa94 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:15.493924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-33b3e157-f12e-4a65-a3fd-a8962491fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:15.505655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1056/2124] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:27:15.513195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3e157-f12e-4a65-a3fd-a8962491fa94 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:15.539484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3e157-f12e-4a65-a3fd-a8962491fa94 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:15.562205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:15.562850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:15.562850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:15.563068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:15.564626 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:27:15.564950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:15.566653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:15.579006 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7caa126f7eab4eef8db03374e82a6ab9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:15.595498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7caa126f7eab4eef8db03374e82a6ab9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:15.596678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7caa126f7eab4eef8db03374e82a6ab9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:15.597667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7caa126f7eab4eef8db03374e82a6ab9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:15.604189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1057/2125] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:27:15.615018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:15.617600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:15.642616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275885b-11a8-4550-8448-8f6a08d29601 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:15.649015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba16e9c-8018-46f0-bb2b-3cba8310debf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:15.650529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba16e9c-8018-46f0-bb2b-3cba8310debf admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:15.650984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba16e9c-8018-46f0-bb2b-3cba8310debf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:15.651555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba16e9c-8018-46f0-bb2b-3cba8310debf admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:15.653922 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba16e9c-8018-46f0-bb2b-3cba8310debf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:15.656236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba16e9c-8018-46f0-bb2b-3cba8310debf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:15.721507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7caa126f7eab4eef8db03374e82a6ab9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:15.740238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:15.765559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1069/2126] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:15 2022] DELETE /identity/v3/users/7caa126f7eab4eef8db03374e82a6ab9 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:15.828672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1058/2127] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:15 2022] POST /identity/v3/projects => generated 367 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:15.831013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:15.940189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240236e-c428-4aba-84cf-123b6105b8d4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:16.080855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c6931-799e-4c39-88e7-8c8cff975ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:16.092952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6931-799e-4c39-88e7-8c8cff975ced admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:16.092952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6931-799e-4c39-88e7-8c8cff975ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:16.092952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6931-799e-4c39-88e7-8c8cff975ced admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:16.092952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c6931-799e-4c39-88e7-8c8cff975ced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:16.092952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c6931-799e-4c39-88e7-8c8cff975ced admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:16.480224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1070/2128] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:15 2022] POST /identity/v3/users => generated 284 bytes in 647 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:16.482710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba16e9c-8018-46f0-bb2b-3cba8310debf admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:16.498475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba16e9c-8018-46f0-bb2b-3cba8310debf admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:16.511079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0340d-b144-4105-93db-7678d9931251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:16.512305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0340d-b144-4105-93db-7678d9931251 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:16.514637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0340d-b144-4105-93db-7678d9931251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:16.514637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0340d-b144-4105-93db-7678d9931251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:16.514637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0340d-b144-4105-93db-7678d9931251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:16.518283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac0340d-b144-4105-93db-7678d9931251 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:16.547858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1059/2129] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:16.549170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c6931-799e-4c39-88e7-8c8cff975ced admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:16.563446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c6931-799e-4c39-88e7-8c8cff975ced admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:16.579342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147029f-dc80-489c-895c-b5d2371fded1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:16.580991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147029f-dc80-489c-895c-b5d2371fded1 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:16.581512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147029f-dc80-489c-895c-b5d2371fded1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:16.582004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147029f-dc80-489c-895c-b5d2371fded1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:16.583085 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147029f-dc80-489c-895c-b5d2371fded1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:16.585127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147029f-dc80-489c-895c-b5d2371fded1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:16.600267 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1071/2130] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:16.605025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0340d-b144-4105-93db-7678d9931251 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:16.635505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147029f-dc80-489c-895c-b5d2371fded1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:16.640714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1147029f-dc80-489c-895c-b5d2371fded1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:16.650146 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0340d-b144-4105-93db-7678d9931251 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:16.673432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161bccee-ec1f-41c7-95d5-53a2099be62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:16.675522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bccee-ec1f-41c7-95d5-53a2099be62a admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:16.675916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bccee-ec1f-41c7-95d5-53a2099be62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:16.676621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161bccee-ec1f-41c7-95d5-53a2099be62a admin admin] PATH_INFO: `/v3/projects/560710d52f2e40b1a48529941b390623/users/e8d92724a3544306a2e5d1e06a06afec/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:16.693108 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147029f-dc80-489c-895c-b5d2371fded1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:16.706854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:16.707882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:16.708267 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:16.709082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bccee-ec1f-41c7-95d5-53a2099be62a admin admin] RBAC: Authorizing `identity:create_grant(project_id=560710d52f2e40b1a48529941b390623, user_id=e8d92724a3544306a2e5d1e06a06afec, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:16.709305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] PATH_INFO: `/v3/projects/9f556df7a95041279685098584415ed1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:16.712008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f556df7a95041279685098584415ed1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:16.712936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161bccee-ec1f-41c7-95d5-53a2099be62a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:16.714271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:16.833119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1060/2131] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:16 2022] PUT /identity/v3/projects/560710d52f2e40b1a48529941b390623/users/e8d92724a3544306a2e5d1e06a06afec/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:16.845108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1298143-18e1-4275-8b4d-fe5733372e25 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:16.845398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1298143-18e1-4275-8b4d-fe5733372e25 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:16.845674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1298143-18e1-4275-8b4d-fe5733372e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:16.927098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d1298143-18e1-4275-8b4d-fe5733372e25 None None] MFA Rules not processed for user `e8d92724a3544306a2e5d1e06a06afec`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:16.928289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d1298143-18e1-4275-8b4d-fe5733372e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:16.999988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:17.005063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f556df7a95041279685098584415ed1', 'request_id': 'req-6cece254-63f9-4461-9719-297dc9ff2e7e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:17.016704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f556df7a95041279685098584415ed1', 'request_id': 'req-6cece254-63f9-4461-9719-297dc9ff2e7e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:17.111012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1061/2132] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:16 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:17.118561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1298143-18e1-4275-8b4d-fe5733372e25 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:17.124687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f556df7a95041279685098584415ed1', 'request_id': 'req-6cece254-63f9-4461-9719-297dc9ff2e7e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:17.149819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] The token cache is being invalidate because project 9f556df7a95041279685098584415ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:17.150465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:17.152334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6cece254-63f9-4461-9719-297dc9ff2e7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:17.155421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1072/2133] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:16 2022] DELETE /identity/v3/projects/9f556df7a95041279685098584415ed1 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:17.167071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f0d58-b547-44d9-a9b6-d30245244d2c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:17.167691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f0d58-b547-44d9-a9b6-d30245244d2c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:17.168220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f0d58-b547-44d9-a9b6-d30245244d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:17.184401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:17.224336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-867f0d58-b547-44d9-a9b6-d30245244d2c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:17.232309 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-867f0d58-b547-44d9-a9b6-d30245244d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:17.339274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1298143-18e1-4275-8b4d-fe5733372e25 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:17.368127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1073/2134] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:17.369783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc44f8-ed6b-4bae-ab20-888e2293d0ac None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:17.370182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc44f8-ed6b-4bae-ab20-888e2293d0ac None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:17.370637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc44f8-ed6b-4bae-ab20-888e2293d0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:17.396707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-35bc44f8-ed6b-4bae-ab20-888e2293d0ac None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:17.400144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-35bc44f8-ed6b-4bae-ab20-888e2293d0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:17.408824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1074/2135] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:17.412896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc44f8-ed6b-4bae-ab20-888e2293d0ac None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:17.423479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc44f8-ed6b-4bae-ab20-888e2293d0ac None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:17.431554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d06f4-84bd-423b-808f-31ae31af074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:17.432333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d06f4-84bd-423b-808f-31ae31af074e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:17.432741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d06f4-84bd-423b-808f-31ae31af074e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:17.433129 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d06f4-84bd-423b-808f-31ae31af074e admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:17.434230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d06f4-84bd-423b-808f-31ae31af074e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:27:17.434721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d06f4-84bd-423b-808f-31ae31af074e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:17.436487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d06f4-84bd-423b-808f-31ae31af074e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:17.456051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58754dc5-a0f1-4851-b898-a2eace9db862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:17.457070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58754dc5-a0f1-4851-b898-a2eace9db862 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:17.457598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58754dc5-a0f1-4851-b898-a2eace9db862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:17.458232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1075/2136] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:27:17.458545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58754dc5-a0f1-4851-b898-a2eace9db862 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:17.459503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58754dc5-a0f1-4851-b898-a2eace9db862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:17.460140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d06f4-84bd-423b-808f-31ae31af074e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:17.462618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58754dc5-a0f1-4851-b898-a2eace9db862 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:17.473755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d06f4-84bd-423b-808f-31ae31af074e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:17.481674 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f073a-cf6c-4536-b6d9-ee4c8afd52ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:17.483015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f073a-cf6c-4536-b6d9-ee4c8afd52ba admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:17.483015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f073a-cf6c-4536-b6d9-ee4c8afd52ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:17.493546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f073a-cf6c-4536-b6d9-ee4c8afd52ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:17.493546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f073a-cf6c-4536-b6d9-ee4c8afd52ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:17.493546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f073a-cf6c-4536-b6d9-ee4c8afd52ba admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:17.758021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1062/2137] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:17 2022] POST /identity/v3/projects => generated 369 bytes in 641 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:17.765496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58754dc5-a0f1-4851-b898-a2eace9db862 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:17.773942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-58754dc5-a0f1-4851-b898-a2eace9db862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:17.786732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1076/2138] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:17 2022] POST /identity/v3/projects => generated 369 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:17.790640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f073a-cf6c-4536-b6d9-ee4c8afd52ba admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:17.934457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:17.962278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58754dc5-a0f1-4851-b898-a2eace9db862 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:17.966033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f073a-cf6c-4536-b6d9-ee4c8afd52ba admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:17.975569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c0d57-3c97-4f3a-9128-81f68d793864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:17.977376 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c0d57-3c97-4f3a-9128-81f68d793864 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:17.978138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c0d57-3c97-4f3a-9128-81f68d793864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:17.979202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9abb35-607f-4b57-8281-009d741277b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:17.979917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9abb35-607f-4b57-8281-009d741277b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:17.980258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9abb35-607f-4b57-8281-009d741277b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:17.982626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c0d57-3c97-4f3a-9128-81f68d793864 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:17.982884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9abb35-607f-4b57-8281-009d741277b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:17.983595 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9abb35-607f-4b57-8281-009d741277b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:17.985127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c0d57-3c97-4f3a-9128-81f68d793864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:17.987914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9abb35-607f-4b57-8281-009d741277b1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:17.999004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c0d57-3c97-4f3a-9128-81f68d793864 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:18.148646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1063/2139] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:17 2022] POST /identity/v3/users => generated 291 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:27:18.156861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9abb35-607f-4b57-8281-009d741277b1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:18.179685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1077/2140] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:17 2022] POST /identity/v3/users => generated 285 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:27:18.181798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c0d57-3c97-4f3a-9128-81f68d793864 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:18.186849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9abb35-607f-4b57-8281-009d741277b1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:18.197891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956056a3-8795-4a9b-b792-5d0cb833954b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:18.200016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956056a3-8795-4a9b-b792-5d0cb833954b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:18.200655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956056a3-8795-4a9b-b792-5d0cb833954b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:18.201070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956056a3-8795-4a9b-b792-5d0cb833954b admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:18.201949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956056a3-8795-4a9b-b792-5d0cb833954b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:18.205915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956056a3-8795-4a9b-b792-5d0cb833954b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:18.231211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c0d57-3c97-4f3a-9128-81f68d793864 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:18.241728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6370-ea12-4e5d-a327-2cdc27edd223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:18.244059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6370-ea12-4e5d-a327-2cdc27edd223 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:18.244874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6370-ea12-4e5d-a327-2cdc27edd223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:18.248090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6370-ea12-4e5d-a327-2cdc27edd223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:18.250756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6370-ea12-4e5d-a327-2cdc27edd223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:18.255362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6370-ea12-4e5d-a327-2cdc27edd223 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:18.257721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1064/2141] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:18.262696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956056a3-8795-4a9b-b792-5d0cb833954b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:18.289072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1078/2142] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:18.289838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956056a3-8795-4a9b-b792-5d0cb833954b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:18.291651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6370-ea12-4e5d-a327-2cdc27edd223 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:18.318010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887225f-3485-4de5-afd1-001773f6ae6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:18.319020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887225f-3485-4de5-afd1-001773f6ae6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:18.319326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887225f-3485-4de5-afd1-001773f6ae6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:18.319595 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887225f-3485-4de5-afd1-001773f6ae6d admin admin] PATH_INFO: `/v3/projects/6f6415338f68474897dff669feb0dc43/users/dbb1a13bd952438a866f2f2009110c36/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:18.320220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6370-ea12-4e5d-a327-2cdc27edd223 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:18.339316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1abcd-d958-4ff6-b0b2-d51ce9bfbf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:18.341424 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1abcd-d958-4ff6-b0b2-d51ce9bfbf56 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:18.341719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1abcd-d958-4ff6-b0b2-d51ce9bfbf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:18.342149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1abcd-d958-4ff6-b0b2-d51ce9bfbf56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:18.343244 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1abcd-d958-4ff6-b0b2-d51ce9bfbf56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:18.349022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1abcd-d958-4ff6-b0b2-d51ce9bfbf56 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:18.352313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887225f-3485-4de5-afd1-001773f6ae6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6415338f68474897dff669feb0dc43, user_id=dbb1a13bd952438a866f2f2009110c36, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:18.357888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887225f-3485-4de5-afd1-001773f6ae6d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:18.382187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1079/2143] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:18.389283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1abcd-d958-4ff6-b0b2-d51ce9bfbf56 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:18.413818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1abcd-d958-4ff6-b0b2-d51ce9bfbf56 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:18.426755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae5adb-74bf-4248-a28c-77b28ff464e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:18.428640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae5adb-74bf-4248-a28c-77b28ff464e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:18.428640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae5adb-74bf-4248-a28c-77b28ff464e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:18.429133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae5adb-74bf-4248-a28c-77b28ff464e0 admin admin] PATH_INFO: `/v3/projects/f8fd6fab9a664be998ad9ba19a9a34a0/users/229582e0a5f1402fbe1711575261e37c/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:18.454878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae5adb-74bf-4248-a28c-77b28ff464e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fd6fab9a664be998ad9ba19a9a34a0, user_id=229582e0a5f1402fbe1711575261e37c, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:18.462218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae5adb-74bf-4248-a28c-77b28ff464e0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:18.487894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1065/2144] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:18 2022] PUT /identity/v3/projects/6f6415338f68474897dff669feb0dc43/users/dbb1a13bd952438a866f2f2009110c36/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:18.493730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887225f-3485-4de5-afd1-001773f6ae6d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:18.615410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1080/2145] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:18 2022] PUT /identity/v3/projects/f8fd6fab9a664be998ad9ba19a9a34a0/users/229582e0a5f1402fbe1711575261e37c/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:18.628536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf8858-92e1-43f6-bb98-f7970405299b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:18.628837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf8858-92e1-43f6-bb98-f7970405299b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:18.631926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf8858-92e1-43f6-bb98-f7970405299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:18.660529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:18.686388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887225f-3485-4de5-afd1-001773f6ae6d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:18.744245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-0bbf8858-92e1-43f6-bb98-f7970405299b None None] MFA Rules not processed for user `229582e0a5f1402fbe1711575261e37c`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:18.747788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0bbf8858-92e1-43f6-bb98-f7970405299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:18.802015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8205a4-de12-4ac8-b18c-1866b4f3d34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:18.804249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8205a4-de12-4ac8-b18c-1866b4f3d34c admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:18.805516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8205a4-de12-4ac8-b18c-1866b4f3d34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:18.806559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8205a4-de12-4ac8-b18c-1866b4f3d34c admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:18.807584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8205a4-de12-4ac8-b18c-1866b4f3d34c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:18.810187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8205a4-de12-4ac8-b18c-1866b4f3d34c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:18.825998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1066/2146] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:18.832688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8205a4-de12-4ac8-b18c-1866b4f3d34c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:18.857465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8205a4-de12-4ac8-b18c-1866b4f3d34c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:18.866382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7307d4-1b37-4052-9229-402f0fa215dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:18.868015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7307d4-1b37-4052-9229-402f0fa215dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:18.868545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7307d4-1b37-4052-9229-402f0fa215dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:18.869076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7307d4-1b37-4052-9229-402f0fa215dc admin admin] PATH_INFO: `/v3/projects/6f6415338f68474897dff669feb0dc43/users/dbb1a13bd952438a866f2f2009110c36/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:18.872503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7307d4-1b37-4052-9229-402f0fa215dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6415338f68474897dff669feb0dc43, user_id=dbb1a13bd952438a866f2f2009110c36, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:18.876884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7307d4-1b37-4052-9229-402f0fa215dc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:18.892972 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1067/2147] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:18 2022] PUT /identity/v3/projects/6f6415338f68474897dff669feb0dc43/users/dbb1a13bd952438a866f2f2009110c36/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:18.916344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013636d-f4eb-400b-bdde-52cd3cdbc9be None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:18.916572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013636d-f4eb-400b-bdde-52cd3cdbc9be None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:18.916795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013636d-f4eb-400b-bdde-52cd3cdbc9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:18.983164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9013636d-f4eb-400b-bdde-52cd3cdbc9be None None] MFA Rules not processed for user `dbb1a13bd952438a866f2f2009110c36`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:18.985653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9013636d-f4eb-400b-bdde-52cd3cdbc9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:19.012198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1081/2148] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:19.028562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf8858-92e1-43f6-bb98-f7970405299b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:19.030074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0bbf8858-92e1-43f6-bb98-f7970405299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:19.064241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:19.167346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1068/2149] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:18 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:19.182965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013636d-f4eb-400b-bdde-52cd3cdbc9be None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:19.186299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9013636d-f4eb-400b-bdde-52cd3cdbc9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:19.202489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf8858-92e1-43f6-bb98-f7970405299b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:19.343551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013636d-f4eb-400b-bdde-52cd3cdbc9be None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:19.359329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a76715-7dfd-465c-9287-4e55a9f7c045 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:19.363346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a76715-7dfd-465c-9287-4e55a9f7c045 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:19.364060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a76715-7dfd-465c-9287-4e55a9f7c045 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:19.364677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a76715-7dfd-465c-9287-4e55a9f7c045 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:19.405551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a76715-7dfd-465c-9287-4e55a9f7c045 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:19.409960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a76715-7dfd-465c-9287-4e55a9f7c045 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:19.411960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05a8d4-f9e6-4901-b3b6-64b8b8898844 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9969632e796345fa9671a45cf61ede3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:19.412994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05a8d4-f9e6-4901-b3b6-64b8b8898844 service glance] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:19.414463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05a8d4-f9e6-4901-b3b6-64b8b8898844 service glance] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:19.415818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05a8d4-f9e6-4901-b3b6-64b8b8898844 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:19.452675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1069/2150] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:19 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:19.585973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05a8d4-f9e6-4901-b3b6-64b8b8898844 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:19.590488 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05a8d4-f9e6-4901-b3b6-64b8b8898844 service glance] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:19.620887 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1082/2151] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:19 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:19.657432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a76715-7dfd-465c-9287-4e55a9f7c045 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:19.659253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c1a76715-7dfd-465c-9287-4e55a9f7c045 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:19.690952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a76715-7dfd-465c-9287-4e55a9f7c045 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:19.700792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c43ae-1ca1-4c17-83d7-b673ffc0adb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:19.703334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c43ae-1ca1-4c17-83d7-b673ffc0adb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:19.704524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c43ae-1ca1-4c17-83d7-b673ffc0adb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:19.705810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c43ae-1ca1-4c17-83d7-b673ffc0adb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:19.713302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-835c43ae-1ca1-4c17-83d7-b673ffc0adb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:19.830162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c43ae-1ca1-4c17-83d7-b673ffc0adb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:19.832762 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c43ae-1ca1-4c17-83d7-b673ffc0adb1 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:19.852561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1070/2152] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:27:27.451292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05a8d4-f9e6-4901-b3b6-64b8b8898844 service glance] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:27.456734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:27.523159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05a8d4-f9e6-4901-b3b6-64b8b8898844 service glance] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:27.531200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:27.533672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:27.533672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:27.533672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] PATH_INFO: `/v3/users/e8d92724a3544306a2e5d1e06a06afec` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:27.534533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d92724a3544306a2e5d1e06a06afec)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:27.536499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:27.865586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d92724a3544306a2e5d1e06a06afec'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:27.875296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d92724a3544306a2e5d1e06a06afec'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:27.876055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d92724a3544306a2e5d1e06a06afec'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:27.876816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d92724a3544306a2e5d1e06a06afec'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:27.959229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-75b68b54-04f8-431b-9fc2-3b1c59a4046f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d92724a3544306a2e5d1e06a06afec'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:27.978337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1083/2153] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:27 2022] DELETE /identity/v3/users/e8d92724a3544306a2e5d1e06a06afec => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:27.984045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e14c0d-9d78-4611-96a5-d44b464436fc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:27.984605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e14c0d-9d78-4611-96a5-d44b464436fc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:27.984605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e14c0d-9d78-4611-96a5-d44b464436fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:28.002464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-90e14c0d-9d78-4611-96a5-d44b464436fc None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:28.003685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-90e14c0d-9d78-4611-96a5-d44b464436fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:28.037933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124ec0d-b17d-470a-94c0-0a5657b40463 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:28.038234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124ec0d-b17d-470a-94c0-0a5657b40463 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:28.038545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124ec0d-b17d-470a-94c0-0a5657b40463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:28.060857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6124ec0d-b17d-470a-94c0-0a5657b40463 None None] MFA Rules not processed for user `3cf7a5b535234632bb00a9b87ceb5354`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:28.063131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6124ec0d-b17d-470a-94c0-0a5657b40463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:28.081995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1071/2154] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:28.092858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e14c0d-9d78-4611-96a5-d44b464436fc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:28.094511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-90e14c0d-9d78-4611-96a5-d44b464436fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:28.102747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:28.130466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1084/2155] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:28.165530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e14c0d-9d78-4611-96a5-d44b464436fc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:28.169314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1178-6e68-4e22-a98a-335c9b689f59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:28.169974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1178-6e68-4e22-a98a-335c9b689f59 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:28.170265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1178-6e68-4e22-a98a-335c9b689f59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:28.170651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1178-6e68-4e22-a98a-335c9b689f59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:28.181263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a1178-6e68-4e22-a98a-335c9b689f59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:28.182863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a1178-6e68-4e22-a98a-335c9b689f59 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:28.195128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1072/2156] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:28.866374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124ec0d-b17d-470a-94c0-0a5657b40463 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:28.869089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6124ec0d-b17d-470a-94c0-0a5657b40463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:28.900911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124ec0d-b17d-470a-94c0-0a5657b40463 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:28.908375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:28.909484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:28.909841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:28.911000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] PATH_INFO: `/v3/projects/560710d52f2e40b1a48529941b390623` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:28.912580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=560710d52f2e40b1a48529941b390623)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:28.914834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:28.945509 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:29.101616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '560710d52f2e40b1a48529941b390623', 'request_id': 'req-55344b23-786c-46f5-8b3b-79ccb30d28a9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.107935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '560710d52f2e40b1a48529941b390623', 'request_id': 'req-55344b23-786c-46f5-8b3b-79ccb30d28a9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.221637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '560710d52f2e40b1a48529941b390623', 'request_id': 'req-55344b23-786c-46f5-8b3b-79ccb30d28a9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.234142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] The token cache is being invalidate because project 560710d52f2e40b1a48529941b390623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:29.234563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.235231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.237002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1085/2157] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:28 2022] DELETE /identity/v3/projects/560710d52f2e40b1a48529941b390623 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:29.241253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1178-6e68-4e22-a98a-335c9b689f59 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:29.252210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1178-6e68-4e22-a98a-335c9b689f59 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:29.321833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:29.322813 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:29.323100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:29.323371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] PATH_INFO: `/v3/users/dbb1a13bd952438a866f2f2009110c36` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:29.325323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbb1a13bd952438a866f2f2009110c36)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:29.327343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:29.678010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbb1a13bd952438a866f2f2009110c36'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.678914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbb1a13bd952438a866f2f2009110c36'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.705633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbb1a13bd952438a866f2f2009110c36'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.706715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbb1a13bd952438a866f2f2009110c36'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.809320 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e9693421-3bc5-4aec-82c6-6a9f9bce87d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbb1a13bd952438a866f2f2009110c36'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:29.833663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1073/2158] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:29 2022] DELETE /identity/v3/users/dbb1a13bd952438a866f2f2009110c36 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:30.416478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:30.419000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:30.455931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55344b23-786c-46f5-8b3b-79ccb30d28a9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:30.519978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:30.520823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:30.521138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:30.521441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] PATH_INFO: `/v3/projects/6f6415338f68474897dff669feb0dc43` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:30.522875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6415338f68474897dff669feb0dc43)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:30.524995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:30.653991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6415338f68474897dff669feb0dc43', 'request_id': 'req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:30.661452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6415338f68474897dff669feb0dc43', 'request_id': 'req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:30.781485 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6415338f68474897dff669feb0dc43', 'request_id': 'req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:30.797954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] The token cache is being invalidate because project 6f6415338f68474897dff669feb0dc43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:30.798372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:30.799230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2f9d2b0f-c8b5-43c7-bbe3-224ec9584e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:30.801857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1086/2159] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:30 2022] DELETE /identity/v3/projects/6f6415338f68474897dff669feb0dc43 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:30.808524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4282d9-51c0-49d0-ae34-49fc3bb0452c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:30.810160 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4282d9-51c0-49d0-ae34-49fc3bb0452c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:30.810669 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4282d9-51c0-49d0-ae34-49fc3bb0452c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:30.840887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2e4282d9-51c0-49d0-ae34-49fc3bb0452c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:30.842722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2e4282d9-51c0-49d0-ae34-49fc3bb0452c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:30.866985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:30.912535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1074/2160] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:30.920835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e8ee8-b2e8-49cd-bb47-cb0b3877d3c7 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:30.921130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e8ee8-b2e8-49cd-bb47-cb0b3877d3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:30.921420 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e8ee8-b2e8-49cd-bb47-cb0b3877d3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:30.942332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-369e8ee8-b2e8-49cd-bb47-cb0b3877d3c7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:30.943587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-369e8ee8-b2e8-49cd-bb47-cb0b3877d3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:30.952421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1087/2161] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:27:30.956691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4282d9-51c0-49d0-ae34-49fc3bb0452c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:30.966636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4282d9-51c0-49d0-ae34-49fc3bb0452c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:30.972081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67908a30-6a82-49ee-b801-38b51c54f079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:30.972755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67908a30-6a82-49ee-b801-38b51c54f079 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:30.973043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67908a30-6a82-49ee-b801-38b51c54f079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:30.973436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67908a30-6a82-49ee-b801-38b51c54f079 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:30.974065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67908a30-6a82-49ee-b801-38b51c54f079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:27:30.974380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67908a30-6a82-49ee-b801-38b51c54f079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:30.976071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67908a30-6a82-49ee-b801-38b51c54f079 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:30.991809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1075/2162] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:27:30.997598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e8ee8-b2e8-49cd-bb47-cb0b3877d3c7 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:31.007353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e8ee8-b2e8-49cd-bb47-cb0b3877d3c7 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:31.013384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1330f69-9ca7-4337-b632-13dd5593206f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:31.014046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1330f69-9ca7-4337-b632-13dd5593206f admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:31.014537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1330f69-9ca7-4337-b632-13dd5593206f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:31.014673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1330f69-9ca7-4337-b632-13dd5593206f admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:31.015370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1330f69-9ca7-4337-b632-13dd5593206f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:31.017495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1330f69-9ca7-4337-b632-13dd5593206f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:31.123312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1088/2163] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:30 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:31.127491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67908a30-6a82-49ee-b801-38b51c54f079 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:31.138548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67908a30-6a82-49ee-b801-38b51c54f079 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:31.216322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76486b-bd34-47b1-a5d3-7b9930a8b672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:31.217077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76486b-bd34-47b1-a5d3-7b9930a8b672 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:31.217409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76486b-bd34-47b1-a5d3-7b9930a8b672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:31.217747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76486b-bd34-47b1-a5d3-7b9930a8b672 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:31.218617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76486b-bd34-47b1-a5d3-7b9930a8b672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:31.220726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76486b-bd34-47b1-a5d3-7b9930a8b672 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:31.384480 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1076/2164] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:31 2022] POST /identity/v3/users => generated 290 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:27:31.385853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1330f69-9ca7-4337-b632-13dd5593206f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:31.396843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1330f69-9ca7-4337-b632-13dd5593206f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:31.401462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9beb7-92ba-44b1-b589-17dea6141eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:31.402445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9beb7-92ba-44b1-b589-17dea6141eac admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:31.402688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9beb7-92ba-44b1-b589-17dea6141eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:31.402971 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9beb7-92ba-44b1-b589-17dea6141eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:31.403708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9beb7-92ba-44b1-b589-17dea6141eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:31.405327 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9beb7-92ba-44b1-b589-17dea6141eac admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:31.421040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1089/2165] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:31.422446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76486b-bd34-47b1-a5d3-7b9930a8b672 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:31.437401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76486b-bd34-47b1-a5d3-7b9930a8b672 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:31.443376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481c51a-7da9-4ba8-a051-3fbe823e3fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:31.444217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481c51a-7da9-4ba8-a051-3fbe823e3fee admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:31.444590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481c51a-7da9-4ba8-a051-3fbe823e3fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:31.446456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481c51a-7da9-4ba8-a051-3fbe823e3fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:31.446456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481c51a-7da9-4ba8-a051-3fbe823e3fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:31.453100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481c51a-7da9-4ba8-a051-3fbe823e3fee admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:31.472985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1077/2166] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:31.474398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9beb7-92ba-44b1-b589-17dea6141eac admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:31.481896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:31.491291 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9beb7-92ba-44b1-b589-17dea6141eac admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:31.509735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9c639-0530-4a5d-b50e-b864243a45a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:31.510042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9c639-0530-4a5d-b50e-b864243a45a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:31.510659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9c639-0530-4a5d-b50e-b864243a45a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:31.510948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9c639-0530-4a5d-b50e-b864243a45a3 admin admin] PATH_INFO: `/v3/projects/dafbdf8a2ac24e83b516da4f0ea53687/users/f01b1d69385c454fa30102aac589590d/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:31.561875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9c639-0530-4a5d-b50e-b864243a45a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dafbdf8a2ac24e83b516da4f0ea53687, user_id=f01b1d69385c454fa30102aac589590d, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:31.568704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9c639-0530-4a5d-b50e-b864243a45a3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:31.772487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1090/2167] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:31 2022] PUT /identity/v3/projects/dafbdf8a2ac24e83b516da4f0ea53687/users/f01b1d69385c454fa30102aac589590d/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:31.785485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aface96-f9f7-4d8b-b4cb-f423e89cfb27 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:31.785910 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aface96-f9f7-4d8b-b4cb-f423e89cfb27 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:31.786249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aface96-f9f7-4d8b-b4cb-f423e89cfb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:31.824968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7aface96-f9f7-4d8b-b4cb-f423e89cfb27 None None] MFA Rules not processed for user `f01b1d69385c454fa30102aac589590d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:31.826191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7aface96-f9f7-4d8b-b4cb-f423e89cfb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:31.912292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1078/2168] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:31.917467 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9c639-0530-4a5d-b50e-b864243a45a3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:32.015582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9c639-0530-4a5d-b50e-b864243a45a3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:32.015582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20409bdc-3a50-4577-a7e0-a761d3dcf748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:32.015582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20409bdc-3a50-4577-a7e0-a761d3dcf748 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:32.015582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20409bdc-3a50-4577-a7e0-a761d3dcf748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:32.016158 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20409bdc-3a50-4577-a7e0-a761d3dcf748 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:32.016158 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20409bdc-3a50-4577-a7e0-a761d3dcf748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:32.016158 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20409bdc-3a50-4577-a7e0-a761d3dcf748 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:32.193442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1091/2169] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:31 2022] POST /identity/v3/projects => generated 381 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:32.197524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aface96-f9f7-4d8b-b4cb-f423e89cfb27 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:32.208727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aface96-f9f7-4d8b-b4cb-f423e89cfb27 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:32.232060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:32.274332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df642a96-5d99-4cd2-b7e4-75d01d5eddee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:32.274958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df642a96-5d99-4cd2-b7e4-75d01d5eddee admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:32.275242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df642a96-5d99-4cd2-b7e4-75d01d5eddee admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:32.275515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df642a96-5d99-4cd2-b7e4-75d01d5eddee admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:32.276417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df642a96-5d99-4cd2-b7e4-75d01d5eddee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:32.278861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df642a96-5d99-4cd2-b7e4-75d01d5eddee admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:32.411076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1079/2170] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:32 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:32.415391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20409bdc-3a50-4577-a7e0-a761d3dcf748 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:32.428399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20409bdc-3a50-4577-a7e0-a761d3dcf748 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:32.433945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89cce8-5695-4863-8065-d96fb6696bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:32.434679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89cce8-5695-4863-8065-d96fb6696bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:32.434963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89cce8-5695-4863-8065-d96fb6696bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:32.435238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89cce8-5695-4863-8065-d96fb6696bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:32.436109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89cce8-5695-4863-8065-d96fb6696bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:32.439642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89cce8-5695-4863-8065-d96fb6696bbf admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:32.452428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1092/2171] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:32.456967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df642a96-5d99-4cd2-b7e4-75d01d5eddee admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:32.469044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df642a96-5d99-4cd2-b7e4-75d01d5eddee admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:32.473694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557760d0-c639-426b-a4b8-2838f81f2e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:32.474640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557760d0-c639-426b-a4b8-2838f81f2e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:32.474904 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557760d0-c639-426b-a4b8-2838f81f2e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:32.475183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557760d0-c639-426b-a4b8-2838f81f2e84 admin admin] PATH_INFO: `/v3/projects/830d5a28b2254f3ca367d0a5d97f6db4/users/b8b64b3aad954923a1632b0c4fe71b33/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:32.495249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557760d0-c639-426b-a4b8-2838f81f2e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830d5a28b2254f3ca367d0a5d97f6db4, user_id=b8b64b3aad954923a1632b0c4fe71b33, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:32.498936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557760d0-c639-426b-a4b8-2838f81f2e84 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:32.681176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1080/2172] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:32 2022] PUT /identity/v3/projects/830d5a28b2254f3ca367d0a5d97f6db4/users/b8b64b3aad954923a1632b0c4fe71b33/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:32.687615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89cce8-5695-4863-8065-d96fb6696bbf admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:32.705980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89cce8-5695-4863-8065-d96fb6696bbf admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:32.781540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ea83-a7fd-4149-9cc4-e88405b7f3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:32.782226 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ea83-a7fd-4149-9cc4-e88405b7f3c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:32.782704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ea83-a7fd-4149-9cc4-e88405b7f3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:32.782984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ea83-a7fd-4149-9cc4-e88405b7f3c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:32.783727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ea83-a7fd-4149-9cc4-e88405b7f3c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:32.785385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ea83-a7fd-4149-9cc4-e88405b7f3c5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:32.796707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1093/2173] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:32.802309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557760d0-c639-426b-a4b8-2838f81f2e84 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:32.816925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557760d0-c639-426b-a4b8-2838f81f2e84 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:32.821681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62bf27-faea-4401-a890-e8f595171a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:32.822692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62bf27-faea-4401-a890-e8f595171a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:32.823066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62bf27-faea-4401-a890-e8f595171a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:32.823436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62bf27-faea-4401-a890-e8f595171a75 admin admin] PATH_INFO: `/v3/projects/830d5a28b2254f3ca367d0a5d97f6db4/users/b8b64b3aad954923a1632b0c4fe71b33/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:32.826417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62bf27-faea-4401-a890-e8f595171a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830d5a28b2254f3ca367d0a5d97f6db4, user_id=b8b64b3aad954923a1632b0c4fe71b33, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:32.829211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62bf27-faea-4401-a890-e8f595171a75 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:32.850035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1081/2174] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:32 2022] PUT /identity/v3/projects/830d5a28b2254f3ca367d0a5d97f6db4/users/b8b64b3aad954923a1632b0c4fe71b33/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:32.864477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe2612-7909-4d09-a74d-642da19bc044 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:32.864815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe2612-7909-4d09-a74d-642da19bc044 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:32.865050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe2612-7909-4d09-a74d-642da19bc044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:32.909435 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-45fe2612-7909-4d09-a74d-642da19bc044 None None] MFA Rules not processed for user `b8b64b3aad954923a1632b0c4fe71b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:32.910915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-45fe2612-7909-4d09-a74d-642da19bc044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:32.963647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:33.005173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1094/2175] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:33.016917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62bf27-faea-4401-a890-e8f595171a75 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:33.024716 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1f62bf27-faea-4401-a890-e8f595171a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:33.099462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62bf27-faea-4401-a890-e8f595171a75 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:33.103597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d149a8-f8a5-4bab-a20e-1f35812df1a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:33.104201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d149a8-f8a5-4bab-a20e-1f35812df1a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:33.104486 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d149a8-f8a5-4bab-a20e-1f35812df1a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:33.105408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d149a8-f8a5-4bab-a20e-1f35812df1a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:33.115823 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d149a8-f8a5-4bab-a20e-1f35812df1a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:33.117372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d149a8-f8a5-4bab-a20e-1f35812df1a0 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:33.128986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1082/2176] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:27:33.841656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe2612-7909-4d09-a74d-642da19bc044 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:33.979315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe2612-7909-4d09-a74d-642da19bc044 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:33.986468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:33.986996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:33.987407 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:33.987790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] PATH_INFO: `/v3/users/f01b1d69385c454fa30102aac589590d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:33.990322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f01b1d69385c454fa30102aac589590d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:33.991562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:34.257860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f01b1d69385c454fa30102aac589590d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:34.269355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f01b1d69385c454fa30102aac589590d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:34.270341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f01b1d69385c454fa30102aac589590d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:34.271083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f01b1d69385c454fa30102aac589590d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:34.302497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80295b-ee7b-4a75-b134-b630898d623c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:34.303026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80295b-ee7b-4a75-b134-b630898d623c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:34.303798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80295b-ee7b-4a75-b134-b630898d623c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:34.335926 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-de80295b-ee7b-4a75-b134-b630898d623c None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:34.338333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-de80295b-ee7b-4a75-b134-b630898d623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:34.360288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-56c8e59b-10be-4e9a-9082-e932a930e40b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f01b1d69385c454fa30102aac589590d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:34.397422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1095/2177] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:33 2022] DELETE /identity/v3/users/f01b1d69385c454fa30102aac589590d => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:34.401811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fcd81-bb52-4672-8be6-b3badec918e1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:34.402717 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fcd81-bb52-4672-8be6-b3badec918e1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:34.406624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fcd81-bb52-4672-8be6-b3badec918e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:34.440136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-299fcd81-bb52-4672-8be6-b3badec918e1 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:34.441606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-299fcd81-bb52-4672-8be6-b3badec918e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:34.479623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1083/2178] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:27:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:34.501395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80295b-ee7b-4a75-b134-b630898d623c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:34.503089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-de80295b-ee7b-4a75-b134-b630898d623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:34.515812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:34.546713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1096/2179] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:27:34.556116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fcd81-bb52-4672-8be6-b3badec918e1 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:34.557870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-299fcd81-bb52-4672-8be6-b3badec918e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:34.559977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:34.620050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80295b-ee7b-4a75-b134-b630898d623c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:34.639897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fcd81-bb52-4672-8be6-b3badec918e1 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:34.644289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b5f41-8c8b-48a2-aeb1-08b77be9be4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:34.645416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b5f41-8c8b-48a2-aeb1-08b77be9be4b service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:34.645416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b5f41-8c8b-48a2-aeb1-08b77be9be4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:34.645930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b5f41-8c8b-48a2-aeb1-08b77be9be4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:34.657371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b5f41-8c8b-48a2-aeb1-08b77be9be4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:34.659866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b5f41-8c8b-48a2-aeb1-08b77be9be4b service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:34.680889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1097/2180] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:34.706707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a817-a8e6-4a79-b791-1007b7274bc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:34.708098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a817-a8e6-4a79-b791-1007b7274bc6 service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:34.708098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a817-a8e6-4a79-b791-1007b7274bc6 service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:34.708098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a817-a8e6-4a79-b791-1007b7274bc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:34.805691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a817-a8e6-4a79-b791-1007b7274bc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:34.808254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a817-a8e6-4a79-b791-1007b7274bc6 service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:34.817818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1084/2181] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:27:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 12:27:35.580609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b5f41-8c8b-48a2-aeb1-08b77be9be4b service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:35.582124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-798b5f41-8c8b-48a2-aeb1-08b77be9be4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:35.623080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b5f41-8c8b-48a2-aeb1-08b77be9be4b service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:35.631649 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:35.635554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:35.636203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:35.644656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] PATH_INFO: `/v3/projects/dafbdf8a2ac24e83b516da4f0ea53687` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:35.649175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dafbdf8a2ac24e83b516da4f0ea53687)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:35.654254 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:35.919247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dafbdf8a2ac24e83b516da4f0ea53687', 'request_id': 'req-52e29ff3-6213-4327-9f5e-540818652351'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:35.931457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dafbdf8a2ac24e83b516da4f0ea53687', 'request_id': 'req-52e29ff3-6213-4327-9f5e-540818652351'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:36.009299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dafbdf8a2ac24e83b516da4f0ea53687', 'request_id': 'req-52e29ff3-6213-4327-9f5e-540818652351'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:36.025215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] The token cache is being invalidate because project dafbdf8a2ac24e83b516da4f0ea53687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:36.025597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:36.026920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:36.029730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1098/2182] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:35 2022] DELETE /identity/v3/projects/dafbdf8a2ac24e83b516da4f0ea53687 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:36.033776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a817-a8e6-4a79-b791-1007b7274bc6 service swift] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:36.048549 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a817-a8e6-4a79-b791-1007b7274bc6 service swift] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:36.139930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:36.140992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:36.141550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:36.142035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] PATH_INFO: `/v3/users/b8b64b3aad954923a1632b0c4fe71b33` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:36.143833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b64b3aad954923a1632b0c4fe71b33)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:36.146048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:36.156877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:36.345051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b64b3aad954923a1632b0c4fe71b33'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:36.346003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b64b3aad954923a1632b0c4fe71b33'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:36.369026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b64b3aad954923a1632b0c4fe71b33'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:36.370057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b64b3aad954923a1632b0c4fe71b33'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:36.460876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f0e2c1b-ba84-4db7-b176-acad09072468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b64b3aad954923a1632b0c4fe71b33'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:36.485359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1085/2183] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:36 2022] DELETE /identity/v3/users/b8b64b3aad954923a1632b0c4fe71b33 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:37.094058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:37.099445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:37.128553 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e29ff3-6213-4327-9f5e-540818652351 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:37.203008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:37.204759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:37.205208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:37.205497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] PATH_INFO: `/v3/projects/830d5a28b2254f3ca367d0a5d97f6db4` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:37.208155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=830d5a28b2254f3ca367d0a5d97f6db4)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:37.210642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:37.241648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:37.343943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830d5a28b2254f3ca367d0a5d97f6db4', 'request_id': 'req-02316de2-0536-4616-9969-940d8069b230'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:37.351545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830d5a28b2254f3ca367d0a5d97f6db4', 'request_id': 'req-02316de2-0536-4616-9969-940d8069b230'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:37.481980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830d5a28b2254f3ca367d0a5d97f6db4', 'request_id': 'req-02316de2-0536-4616-9969-940d8069b230'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:37.504020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] The token cache is being invalidate because project 830d5a28b2254f3ca367d0a5d97f6db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:37.504482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:37.505760 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-02316de2-0536-4616-9969-940d8069b230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:37.508560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1099/2184] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:37 2022] DELETE /identity/v3/projects/830d5a28b2254f3ca367d0a5d97f6db4 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:37.516613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430d3d2-0ea0-4149-92aa-25c925978b26 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:37.517160 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430d3d2-0ea0-4149-92aa-25c925978b26 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:37.517802 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430d3d2-0ea0-4149-92aa-25c925978b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:37.551965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8430d3d2-0ea0-4149-92aa-25c925978b26 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:37.554550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8430d3d2-0ea0-4149-92aa-25c925978b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:37.655889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1086/2185] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:37.665153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673ad7c-6d1c-49f5-9f24-adf82fd5bbbd None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:37.665441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673ad7c-6d1c-49f5-9f24-adf82fd5bbbd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:37.665734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673ad7c-6d1c-49f5-9f24-adf82fd5bbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:37.687951 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e673ad7c-6d1c-49f5-9f24-adf82fd5bbbd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:37.689320 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e673ad7c-6d1c-49f5-9f24-adf82fd5bbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:37.715556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1100/2186] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:37.717921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430d3d2-0ea0-4149-92aa-25c925978b26 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:37.739217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430d3d2-0ea0-4149-92aa-25c925978b26 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:37.750459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103e41a-db5a-413b-b0fc-748c25629a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:37.750459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103e41a-db5a-413b-b0fc-748c25629a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:37.750699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103e41a-db5a-413b-b0fc-748c25629a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:37.752369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103e41a-db5a-413b-b0fc-748c25629a95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:37.753518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103e41a-db5a-413b-b0fc-748c25629a95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:27:37.753937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103e41a-db5a-413b-b0fc-748c25629a95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:37.756497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103e41a-db5a-413b-b0fc-748c25629a95 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:37.775564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673ad7c-6d1c-49f5-9f24-adf82fd5bbbd None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:37.775705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1087/2187] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:27:37.797590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673ad7c-6d1c-49f5-9f24-adf82fd5bbbd None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:37.801086 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98559243-49d2-4f4b-b806-4979a22bdc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:37.802131 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98559243-49d2-4f4b-b806-4979a22bdc1a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:37.802442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98559243-49d2-4f4b-b806-4979a22bdc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:37.802739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98559243-49d2-4f4b-b806-4979a22bdc1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:37.803452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98559243-49d2-4f4b-b806-4979a22bdc1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:37.805510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98559243-49d2-4f4b-b806-4979a22bdc1a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:37.924693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1101/2188] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:37 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:27:37.932978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103e41a-db5a-413b-b0fc-748c25629a95 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:37.954177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103e41a-db5a-413b-b0fc-748c25629a95 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:38.066753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c417ba-9f30-4e01-a4f3-7bd793325db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:38.067579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c417ba-9f30-4e01-a4f3-7bd793325db8 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:38.067870 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c417ba-9f30-4e01-a4f3-7bd793325db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:38.068149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c417ba-9f30-4e01-a4f3-7bd793325db8 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:38.068999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c417ba-9f30-4e01-a4f3-7bd793325db8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:38.071456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c417ba-9f30-4e01-a4f3-7bd793325db8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:38.220979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1088/2189] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:37 2022] POST /identity/v3/users => generated 292 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:38.225802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98559243-49d2-4f4b-b806-4979a22bdc1a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:38.259487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98559243-49d2-4f4b-b806-4979a22bdc1a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:38.268689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f36390-2951-4a08-910c-2dd63e081be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:38.270218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f36390-2951-4a08-910c-2dd63e081be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:38.270941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f36390-2951-4a08-910c-2dd63e081be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:38.271698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f36390-2951-4a08-910c-2dd63e081be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:38.273385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f36390-2951-4a08-910c-2dd63e081be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:38.277492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f36390-2951-4a08-910c-2dd63e081be9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:38.293497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1102/2190] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:38.297499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c417ba-9f30-4e01-a4f3-7bd793325db8 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:38.303225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:38.310023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c417ba-9f30-4e01-a4f3-7bd793325db8 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:38.315066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779573be-2064-478e-9635-ff097a967f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:38.315757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779573be-2064-478e-9635-ff097a967f5e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:38.316047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779573be-2064-478e-9635-ff097a967f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:38.316325 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779573be-2064-478e-9635-ff097a967f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:38.317069 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779573be-2064-478e-9635-ff097a967f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:38.318798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779573be-2064-478e-9635-ff097a967f5e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:38.332170 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1089/2191] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:38.336640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f36390-2951-4a08-910c-2dd63e081be9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:38.352393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f36390-2951-4a08-910c-2dd63e081be9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:38.359109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e3a38-857c-4242-8113-d320fbeaf1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:38.360017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e3a38-857c-4242-8113-d320fbeaf1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:38.360306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e3a38-857c-4242-8113-d320fbeaf1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:38.360585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e3a38-857c-4242-8113-d320fbeaf1d4 admin admin] PATH_INFO: `/v3/projects/fe02c98ccfa444e78a8b256297f8fdc7/users/7beedb2e390843e18dd4972a4be04147/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:38.378616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e3a38-857c-4242-8113-d320fbeaf1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe02c98ccfa444e78a8b256297f8fdc7, user_id=7beedb2e390843e18dd4972a4be04147, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:38.381378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e3a38-857c-4242-8113-d320fbeaf1d4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:38.482186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1103/2192] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:38 2022] PUT /identity/v3/projects/fe02c98ccfa444e78a8b256297f8fdc7/users/7beedb2e390843e18dd4972a4be04147/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:38.484587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60c94e-f15a-440d-89e0-e3e6f626d89f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:38.485173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60c94e-f15a-440d-89e0-e3e6f626d89f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:38.486149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60c94e-f15a-440d-89e0-e3e6f626d89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:38.562986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cc60c94e-f15a-440d-89e0-e3e6f626d89f None None] MFA Rules not processed for user `7beedb2e390843e18dd4972a4be04147`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:38.564243 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cc60c94e-f15a-440d-89e0-e3e6f626d89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:38.709180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1090/2193] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:38.715037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e3a38-857c-4242-8113-d320fbeaf1d4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:38.874919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e3a38-857c-4242-8113-d320fbeaf1d4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:38.874919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0158f-5770-46ae-903b-7e7476b87d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:38.874919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0158f-5770-46ae-903b-7e7476b87d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:38.874919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0158f-5770-46ae-903b-7e7476b87d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:38.875311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0158f-5770-46ae-903b-7e7476b87d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:38.875311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0158f-5770-46ae-903b-7e7476b87d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:38.875311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0158f-5770-46ae-903b-7e7476b87d3d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:39.492783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1104/2194] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:38 2022] POST /identity/v3/projects => generated 385 bytes in 780 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:39.498906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60c94e-f15a-440d-89e0-e3e6f626d89f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:39.511150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60c94e-f15a-440d-89e0-e3e6f626d89f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:39.609701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551df8c-789f-4070-a75a-4b3bf274cd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:39.615705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551df8c-789f-4070-a75a-4b3bf274cd38 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:39.615705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551df8c-789f-4070-a75a-4b3bf274cd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:39.615705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551df8c-789f-4070-a75a-4b3bf274cd38 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:39.615705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551df8c-789f-4070-a75a-4b3bf274cd38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:39.617389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551df8c-789f-4070-a75a-4b3bf274cd38 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:39.746269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1091/2195] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:39 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:39.751050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0158f-5770-46ae-903b-7e7476b87d3d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:39.762986 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0158f-5770-46ae-903b-7e7476b87d3d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:39.770989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82811f2a-85b2-4ec3-bbf0-1558e0653c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:39.771678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82811f2a-85b2-4ec3-bbf0-1558e0653c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:39.771961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82811f2a-85b2-4ec3-bbf0-1558e0653c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:39.772233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82811f2a-85b2-4ec3-bbf0-1558e0653c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:39.772970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82811f2a-85b2-4ec3-bbf0-1558e0653c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:39.774760 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82811f2a-85b2-4ec3-bbf0-1558e0653c59 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:39.787414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1105/2196] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:39.792690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551df8c-789f-4070-a75a-4b3bf274cd38 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:39.806136 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551df8c-789f-4070-a75a-4b3bf274cd38 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:39.816091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806af511-3d50-44fa-b093-9fa700944f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:39.817137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806af511-3d50-44fa-b093-9fa700944f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:39.817432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806af511-3d50-44fa-b093-9fa700944f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:39.817706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806af511-3d50-44fa-b093-9fa700944f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:39.818494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806af511-3d50-44fa-b093-9fa700944f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:39.820334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806af511-3d50-44fa-b093-9fa700944f8b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:39.845107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1092/2197] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:39.845372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82811f2a-85b2-4ec3-bbf0-1558e0653c59 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:39.857626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82811f2a-85b2-4ec3-bbf0-1558e0653c59 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:39.865072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cdc93-e4a3-435f-9e42-4c3e7005ad45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:39.868237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cdc93-e4a3-435f-9e42-4c3e7005ad45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:39.868237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cdc93-e4a3-435f-9e42-4c3e7005ad45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:39.868237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cdc93-e4a3-435f-9e42-4c3e7005ad45 admin admin] PATH_INFO: `/v3/projects/d6b87b3c4241473290448118ca530368/users/bf6e4feaf4a146979574a5f291ae076e/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:39.901118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cdc93-e4a3-435f-9e42-4c3e7005ad45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b87b3c4241473290448118ca530368, user_id=bf6e4feaf4a146979574a5f291ae076e, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:39.904965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cdc93-e4a3-435f-9e42-4c3e7005ad45 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:39.909005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:39.975689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1106/2198] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:39 2022] PUT /identity/v3/projects/d6b87b3c4241473290448118ca530368/users/bf6e4feaf4a146979574a5f291ae076e/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:39.987412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dac99-1960-438d-94f6-b1aed0a5e87b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:39.987841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dac99-1960-438d-94f6-b1aed0a5e87b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:39.988278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dac99-1960-438d-94f6-b1aed0a5e87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:40.018799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5b2dac99-1960-438d-94f6-b1aed0a5e87b None None] MFA Rules not processed for user `bf6e4feaf4a146979574a5f291ae076e`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:40.020015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5b2dac99-1960-438d-94f6-b1aed0a5e87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:40.077287 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:40.103970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1093/2199] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:40.108768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cdc93-e4a3-435f-9e42-4c3e7005ad45 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:40.185386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cdc93-e4a3-435f-9e42-4c3e7005ad45 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:40.190839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e30d2-a890-4817-af59-5c182c6ba3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:40.191800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e30d2-a890-4817-af59-5c182c6ba3cc admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:40.192089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e30d2-a890-4817-af59-5c182c6ba3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:40.192366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e30d2-a890-4817-af59-5c182c6ba3cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:40.193096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e30d2-a890-4817-af59-5c182c6ba3cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:40.195105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e30d2-a890-4817-af59-5c182c6ba3cc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:40.335956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1107/2200] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:40 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:40.341008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dac99-1960-438d-94f6-b1aed0a5e87b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:40.352950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dac99-1960-438d-94f6-b1aed0a5e87b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:40.444275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c9b78-904f-49a7-a6f1-cd5671b07a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:40.445642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c9b78-904f-49a7-a6f1-cd5671b07a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:40.445889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c9b78-904f-49a7-a6f1-cd5671b07a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:40.446252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c9b78-904f-49a7-a6f1-cd5671b07a41 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:40.447182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c9b78-904f-49a7-a6f1-cd5671b07a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:40.450355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c9b78-904f-49a7-a6f1-cd5671b07a41 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:40.567462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1094/2201] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:40 2022] POST /identity/v3/users => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:40.572626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e30d2-a890-4817-af59-5c182c6ba3cc admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:40.588494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e30d2-a890-4817-af59-5c182c6ba3cc admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:40.593367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f7f70-bfee-454d-9a0d-651b9417ca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:40.594060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f7f70-bfee-454d-9a0d-651b9417ca93 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:40.594350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f7f70-bfee-454d-9a0d-651b9417ca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:40.594675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f7f70-bfee-454d-9a0d-651b9417ca93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:40.595478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f7f70-bfee-454d-9a0d-651b9417ca93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:40.597071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f7f70-bfee-454d-9a0d-651b9417ca93 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:40.610070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1108/2202] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:40.614285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c9b78-904f-49a7-a6f1-cd5671b07a41 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:40.627741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c9b78-904f-49a7-a6f1-cd5671b07a41 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:40.633469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8952619-6f92-458e-866f-c034ccc0c9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:40.634250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8952619-6f92-458e-866f-c034ccc0c9fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:40.634627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8952619-6f92-458e-866f-c034ccc0c9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:40.639987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8952619-6f92-458e-866f-c034ccc0c9fa admin admin] PATH_INFO: `/v3/projects/45dc98101c5e490780b53bc2ed1c4f24/users/074217e4b5be47388fbe9a17ac9d71d6/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:40.655355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8952619-6f92-458e-866f-c034ccc0c9fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=45dc98101c5e490780b53bc2ed1c4f24, user_id=074217e4b5be47388fbe9a17ac9d71d6, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:40.658745 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8952619-6f92-458e-866f-c034ccc0c9fa admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:40.800548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1095/2203] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:40 2022] PUT /identity/v3/projects/45dc98101c5e490780b53bc2ed1c4f24/users/074217e4b5be47388fbe9a17ac9d71d6/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:40.808003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f7f70-bfee-454d-9a0d-651b9417ca93 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:40.819854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f7f70-bfee-454d-9a0d-651b9417ca93 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:40.894574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c306f-d87a-4b4b-85cb-bb7efe20dd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:40.895345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c306f-d87a-4b4b-85cb-bb7efe20dd72 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:40.895637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c306f-d87a-4b4b-85cb-bb7efe20dd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:40.895914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c306f-d87a-4b4b-85cb-bb7efe20dd72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:40.896769 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c306f-d87a-4b4b-85cb-bb7efe20dd72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:40.898401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c306f-d87a-4b4b-85cb-bb7efe20dd72 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:40.914112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1109/2204] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:27:40.918266 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8952619-6f92-458e-866f-c034ccc0c9fa admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:40.932644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8952619-6f92-458e-866f-c034ccc0c9fa admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:40.937849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb614172-28d9-4d88-8f5a-297a9f906bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:40.938795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb614172-28d9-4d88-8f5a-297a9f906bae admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:40.939149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb614172-28d9-4d88-8f5a-297a9f906bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:40.939385 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb614172-28d9-4d88-8f5a-297a9f906bae admin admin] PATH_INFO: `/v3/projects/45dc98101c5e490780b53bc2ed1c4f24/users/074217e4b5be47388fbe9a17ac9d71d6/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:40.941612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb614172-28d9-4d88-8f5a-297a9f906bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=45dc98101c5e490780b53bc2ed1c4f24, user_id=074217e4b5be47388fbe9a17ac9d71d6, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:40.944450 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb614172-28d9-4d88-8f5a-297a9f906bae admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:40.969909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1096/2205] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:40 2022] PUT /identity/v3/projects/45dc98101c5e490780b53bc2ed1c4f24/users/074217e4b5be47388fbe9a17ac9d71d6/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:40.972993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498a9dd-25b8-43e7-b4ea-dd037c60933b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:40.973352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498a9dd-25b8-43e7-b4ea-dd037c60933b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:40.973637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c498a9dd-25b8-43e7-b4ea-dd037c60933b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:41.034836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c498a9dd-25b8-43e7-b4ea-dd037c60933b None None] MFA Rules not processed for user `074217e4b5be47388fbe9a17ac9d71d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:41.036666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c498a9dd-25b8-43e7-b4ea-dd037c60933b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:41.103087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:41.121966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1110/2206] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:41.141701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb614172-28d9-4d88-8f5a-297a9f906bae admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:41.143622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cb614172-28d9-4d88-8f5a-297a9f906bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:41.213973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb614172-28d9-4d88-8f5a-297a9f906bae admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:41.222008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23f180-bf58-4542-bb67-7f54081d35c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:41.231391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23f180-bf58-4542-bb67-7f54081d35c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:41.231391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23f180-bf58-4542-bb67-7f54081d35c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:41.231391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23f180-bf58-4542-bb67-7f54081d35c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:41.318324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23f180-bf58-4542-bb67-7f54081d35c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:41.320390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23f180-bf58-4542-bb67-7f54081d35c9 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:41.325108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:41.337138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1097/2207] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:44.816831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498a9dd-25b8-43e7-b4ea-dd037c60933b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:44.827117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c498a9dd-25b8-43e7-b4ea-dd037c60933b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:44.830692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b82ffe-190f-4683-bbf1-70a38a1528bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:44.831337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b82ffe-190f-4683-bbf1-70a38a1528bb service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:44.831619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b82ffe-190f-4683-bbf1-70a38a1528bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:44.831896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b82ffe-190f-4683-bbf1-70a38a1528bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:44.897338 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b82ffe-190f-4683-bbf1-70a38a1528bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:44.899027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b82ffe-190f-4683-bbf1-70a38a1528bb service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:44.908982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1111/2208] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:46.816512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23f180-bf58-4542-bb67-7f54081d35c9 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:46.825620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23f180-bf58-4542-bb67-7f54081d35c9 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:46.829341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e0192-a03c-44db-b3be-674bff778092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:46.830079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e0192-a03c-44db-b3be-674bff778092 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:46.830590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e0192-a03c-44db-b3be-674bff778092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:46.830752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e0192-a03c-44db-b3be-674bff778092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:46.841154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e0192-a03c-44db-b3be-674bff778092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:46.842635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e0192-a03c-44db-b3be-674bff778092 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:46.853436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1098/2209] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:53.062726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b82ffe-190f-4683-bbf1-70a38a1528bb service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:53.140163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b82ffe-190f-4683-bbf1-70a38a1528bb service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:53.145450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:53.146345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:53.155484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:53.155484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] PATH_INFO: `/v3/users/7beedb2e390843e18dd4972a4be04147` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:53.155484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7beedb2e390843e18dd4972a4be04147)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:53.155484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:53.404824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7beedb2e390843e18dd4972a4be04147'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:53.412693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7beedb2e390843e18dd4972a4be04147'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:53.413441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7beedb2e390843e18dd4972a4be04147'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:53.414098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7beedb2e390843e18dd4972a4be04147'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:53.484538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7beedb2e390843e18dd4972a4be04147'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:53.515059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1112/2210] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:53 2022] DELETE /identity/v3/users/7beedb2e390843e18dd4972a4be04147 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:53.519394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60ed10-5945-41ec-b550-afa30776f9d6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:53.519949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60ed10-5945-41ec-b550-afa30776f9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:53.520501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60ed10-5945-41ec-b550-afa30776f9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:53.542456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2c60ed10-5945-41ec-b550-afa30776f9d6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:53.544522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2c60ed10-5945-41ec-b550-afa30776f9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:53.615038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1099/2211] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:53.625886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:53.627684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:53.639351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:53.689473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d33530-67b0-48a9-bed4-3286816984f6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:53.693896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7be67-3451-47b5-8905-ac5be6f242a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:53.702735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7be67-3451-47b5-8905-ac5be6f242a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:53.702735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7be67-3451-47b5-8905-ac5be6f242a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:53.702735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7be67-3451-47b5-8905-ac5be6f242a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:53.705426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7be67-3451-47b5-8905-ac5be6f242a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:53.706917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7be67-3451-47b5-8905-ac5be6f242a2 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:53.717933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1113/2212] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:27:54.476048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60ed10-5945-41ec-b550-afa30776f9d6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:54.478273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2c60ed10-5945-41ec-b550-afa30776f9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:54.501878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60ed10-5945-41ec-b550-afa30776f9d6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:54.507445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:54.508535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:54.508873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:54.509260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] PATH_INFO: `/v3/projects/fe02c98ccfa444e78a8b256297f8fdc7` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:54.510755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe02c98ccfa444e78a8b256297f8fdc7)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:54.515992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:54.757284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe02c98ccfa444e78a8b256297f8fdc7', 'request_id': 'req-62b36ea9-3961-4321-8ce6-9321e62f47d7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:54.766519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe02c98ccfa444e78a8b256297f8fdc7', 'request_id': 'req-62b36ea9-3961-4321-8ce6-9321e62f47d7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:54.895149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe02c98ccfa444e78a8b256297f8fdc7', 'request_id': 'req-62b36ea9-3961-4321-8ce6-9321e62f47d7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:54.909351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] The token cache is being invalidate because project fe02c98ccfa444e78a8b256297f8fdc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:54.909637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:54.910461 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:54.912055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1100/2213] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:54 2022] DELETE /identity/v3/projects/fe02c98ccfa444e78a8b256297f8fdc7 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:27:54.917159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7be67-3451-47b5-8905-ac5be6f242a2 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:54.932584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7be67-3451-47b5-8905-ac5be6f242a2 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:55.008211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:55.009255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:55.009632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:55.010023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] PATH_INFO: `/v3/users/bf6e4feaf4a146979574a5f291ae076e` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:55.012253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6e4feaf4a146979574a5f291ae076e)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:55.014685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:55.315883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6e4feaf4a146979574a5f291ae076e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:55.323323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6e4feaf4a146979574a5f291ae076e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:55.324652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6e4feaf4a146979574a5f291ae076e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:55.325207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6e4feaf4a146979574a5f291ae076e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:55.392026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6e4feaf4a146979574a5f291ae076e'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:55.424315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1114/2214] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:54 2022] DELETE /identity/v3/users/bf6e4feaf4a146979574a5f291ae076e => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:55.981521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:55.983788 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:55.997953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b36ea9-3961-4321-8ce6-9321e62f47d7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:56.075918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:56.077465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:56.078130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:56.078644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] PATH_INFO: `/v3/projects/d6b87b3c4241473290448118ca530368` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:56.088137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b87b3c4241473290448118ca530368)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:56.088137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:56.098827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:56.214493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b87b3c4241473290448118ca530368', 'request_id': 'req-6f0efe33-760b-47d7-a4fa-ef43919f8896'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.220634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b87b3c4241473290448118ca530368', 'request_id': 'req-6f0efe33-760b-47d7-a4fa-ef43919f8896'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.329107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b87b3c4241473290448118ca530368', 'request_id': 'req-6f0efe33-760b-47d7-a4fa-ef43919f8896'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.341768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] The token cache is being invalidate because project d6b87b3c4241473290448118ca530368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:56.349198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.349198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.349198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1101/2215] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:55 2022] DELETE /identity/v3/projects/d6b87b3c4241473290448118ca530368 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:56.349518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:56.350037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:56.363362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef1bde-b91e-45c8-be38-c1fd21880bae admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:56.397647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:56.428854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:56.429754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:56.430038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:56.430313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] PATH_INFO: `/v3/users/074217e4b5be47388fbe9a17ac9d71d6` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:56.431562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=074217e4b5be47388fbe9a17ac9d71d6)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:56.434100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:56.664397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '074217e4b5be47388fbe9a17ac9d71d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.673171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '074217e4b5be47388fbe9a17ac9d71d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.673880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '074217e4b5be47388fbe9a17ac9d71d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.674502 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '074217e4b5be47388fbe9a17ac9d71d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.780601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-52698cfb-3523-466c-b1eb-137e93387af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '074217e4b5be47388fbe9a17ac9d71d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:56.805266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1115/2216] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:27:56 2022] DELETE /identity/v3/users/074217e4b5be47388fbe9a17ac9d71d6 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:57.297112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:57.298817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:57.313110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0efe33-760b-47d7-a4fa-ef43919f8896 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:57.392464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:57.393357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:57.393648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:57.393928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] PATH_INFO: `/v3/projects/45dc98101c5e490780b53bc2ed1c4f24` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:57.395215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=45dc98101c5e490780b53bc2ed1c4f24)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:57.397319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:57.516190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45dc98101c5e490780b53bc2ed1c4f24', 'request_id': 'req-7169aeac-bb93-4b38-be70-67d17d4f4c5d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:57.522458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45dc98101c5e490780b53bc2ed1c4f24', 'request_id': 'req-7169aeac-bb93-4b38-be70-67d17d4f4c5d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:57.643114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45dc98101c5e490780b53bc2ed1c4f24', 'request_id': 'req-7169aeac-bb93-4b38-be70-67d17d4f4c5d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:57.656689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] The token cache is being invalidate because project 45dc98101c5e490780b53bc2ed1c4f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:27:57.656977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:57.657646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7169aeac-bb93-4b38-be70-67d17d4f4c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:27:57.659435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1102/2217] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:27:57 2022] DELETE /identity/v3/projects/45dc98101c5e490780b53bc2ed1c4f24 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:57.664521 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0701ed4-c38e-4b7e-b041-e311e7845170 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:57.665045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0701ed4-c38e-4b7e-b041-e311e7845170 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:57.665157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0701ed4-c38e-4b7e-b041-e311e7845170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:57.685538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e0701ed4-c38e-4b7e-b041-e311e7845170 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:57.686847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e0701ed4-c38e-4b7e-b041-e311e7845170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:57.754791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1116/2218] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:27:57.763738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51def98-8b42-4f48-8267-19c5ab0a70d6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:57.764239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51def98-8b42-4f48-8267-19c5ab0a70d6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:57.764725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51def98-8b42-4f48-8267-19c5ab0a70d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:57.798675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b51def98-8b42-4f48-8267-19c5ab0a70d6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:57.800363 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b51def98-8b42-4f48-8267-19c5ab0a70d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:57.814252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1103/2219] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:27:57.817713 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0701ed4-c38e-4b7e-b041-e311e7845170 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:57.838627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0701ed4-c38e-4b7e-b041-e311e7845170 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:57.846821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd3e0c-6790-42c9-90f6-7361d3bf0a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:57.847819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3e0c-6790-42c9-90f6-7361d3bf0a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:57.848222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3e0c-6790-42c9-90f6-7361d3bf0a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:57.848706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3e0c-6790-42c9-90f6-7361d3bf0a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:57.850066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd3e0c-6790-42c9-90f6-7361d3bf0a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:27:57.850628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd3e0c-6790-42c9-90f6-7361d3bf0a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:57.852738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd3e0c-6790-42c9-90f6-7361d3bf0a46 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:57.866904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1117/2220] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:27:57.873510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51def98-8b42-4f48-8267-19c5ab0a70d6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:57.883889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51def98-8b42-4f48-8267-19c5ab0a70d6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:57.890979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeada95-e93c-48ad-91e4-6f737d4bc904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:57.891692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeada95-e93c-48ad-91e4-6f737d4bc904 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:57.891980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeada95-e93c-48ad-91e4-6f737d4bc904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:57.892277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeada95-e93c-48ad-91e4-6f737d4bc904 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:57.892989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeada95-e93c-48ad-91e4-6f737d4bc904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:57.895038 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeada95-e93c-48ad-91e4-6f737d4bc904 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:58.044408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:58.052049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1104/2221] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:27:57 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:58.056599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd3e0c-6790-42c9-90f6-7361d3bf0a46 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:58.070393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd3e0c-6790-42c9-90f6-7361d3bf0a46 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:58.113202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:27:58.150456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0a813-fda7-4a29-94d5-077a76271eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:58.150456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0a813-fda7-4a29-94d5-077a76271eff admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:58.150456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0a813-fda7-4a29-94d5-077a76271eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:58.150456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0a813-fda7-4a29-94d5-077a76271eff admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:58.150880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0a813-fda7-4a29-94d5-077a76271eff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:58.150880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0a813-fda7-4a29-94d5-077a76271eff admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:58.323418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1118/2222] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:27:58 2022] POST /identity/v3/users => generated 288 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:27:58.327956 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeada95-e93c-48ad-91e4-6f737d4bc904 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:58.339153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeada95-e93c-48ad-91e4-6f737d4bc904 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:58.344535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcc39f-a178-4e6a-9319-94cc10b4b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:58.345310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcc39f-a178-4e6a-9319-94cc10b4b05a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:58.345590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcc39f-a178-4e6a-9319-94cc10b4b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:58.345865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcc39f-a178-4e6a-9319-94cc10b4b05a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:58.350299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcc39f-a178-4e6a-9319-94cc10b4b05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:58.350299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcc39f-a178-4e6a-9319-94cc10b4b05a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:58.363638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1105/2223] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:27:58.367964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0a813-fda7-4a29-94d5-077a76271eff admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:58.378917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0a813-fda7-4a29-94d5-077a76271eff admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:58.383589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53f7ab-f3db-4cbd-85d0-dc865bfd02f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:58.384277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53f7ab-f3db-4cbd-85d0-dc865bfd02f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:58.384544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53f7ab-f3db-4cbd-85d0-dc865bfd02f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:58.384819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53f7ab-f3db-4cbd-85d0-dc865bfd02f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:58.385547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53f7ab-f3db-4cbd-85d0-dc865bfd02f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:58.387372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53f7ab-f3db-4cbd-85d0-dc865bfd02f4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:58.401787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1119/2224] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:27:58.407305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcc39f-a178-4e6a-9319-94cc10b4b05a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:58.415527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcc39f-a178-4e6a-9319-94cc10b4b05a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:58.421468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738adba-2311-40af-8c0c-464bea28ccb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:58.422349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738adba-2311-40af-8c0c-464bea28ccb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:58.422670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738adba-2311-40af-8c0c-464bea28ccb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:58.422941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738adba-2311-40af-8c0c-464bea28ccb2 admin admin] PATH_INFO: `/v3/projects/566bbc906fb54ffab4964acccd8e68d8/users/cc67af013a0c4299acd7f2b4de73f7fa/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:58.440628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738adba-2311-40af-8c0c-464bea28ccb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=566bbc906fb54ffab4964acccd8e68d8, user_id=cc67af013a0c4299acd7f2b4de73f7fa, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:58.443378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738adba-2311-40af-8c0c-464bea28ccb2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:58.561068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1106/2225] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:27:58 2022] PUT /identity/v3/projects/566bbc906fb54ffab4964acccd8e68d8/users/cc67af013a0c4299acd7f2b4de73f7fa/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:27:58.573669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71401b-4d45-4cdb-8389-67be89fa485c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:58.573983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71401b-4d45-4cdb-8389-67be89fa485c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:58.574266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71401b-4d45-4cdb-8389-67be89fa485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:58.604672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cc71401b-4d45-4cdb-8389-67be89fa485c None None] MFA Rules not processed for user `cc67af013a0c4299acd7f2b4de73f7fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:27:58.606462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cc71401b-4d45-4cdb-8389-67be89fa485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:58.679586 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1120/2226] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:27:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:27:58.691293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738adba-2311-40af-8c0c-464bea28ccb2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:27:58.693012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1738adba-2311-40af-8c0c-464bea28ccb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:27:58.753583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738adba-2311-40af-8c0c-464bea28ccb2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:27:58.757409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49defc3-7fae-4286-8f22-15c3260d8816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:27:58.758064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49defc3-7fae-4286-8f22-15c3260d8816 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:27:58.758345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49defc3-7fae-4286-8f22-15c3260d8816 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:27:58.758661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49defc3-7fae-4286-8f22-15c3260d8816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:27:58.769485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49defc3-7fae-4286-8f22-15c3260d8816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:27:58.772230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49defc3-7fae-4286-8f22-15c3260d8816 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:27:58.783985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1107/2227] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:27:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:03.960258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcd111-cc5b-4c47-96f1-6d6b40e0b77d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:03.960879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcd111-cc5b-4c47-96f1-6d6b40e0b77d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:03.960879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcd111-cc5b-4c47-96f1-6d6b40e0b77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:03.982716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3adcd111-cc5b-4c47-96f1-6d6b40e0b77d None None] MFA Rules not processed for user `cc67af013a0c4299acd7f2b4de73f7fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:03.985691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3adcd111-cc5b-4c47-96f1-6d6b40e0b77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:04.003598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1121/2228] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:28:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:08.779384 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:08.880494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49defc3-7fae-4286-8f22-15c3260d8816 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:08.882014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e49defc3-7fae-4286-8f22-15c3260d8816 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:08.952025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49defc3-7fae-4286-8f22-15c3260d8816 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:08.958198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:08.959123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:08.959409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:08.959690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] PATH_INFO: `/v3/users/229582e0a5f1402fbe1711575261e37c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:08.961299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=229582e0a5f1402fbe1711575261e37c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:08.963375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:09.231705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229582e0a5f1402fbe1711575261e37c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:09.232412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229582e0a5f1402fbe1711575261e37c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:09.244978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229582e0a5f1402fbe1711575261e37c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:09.245851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229582e0a5f1402fbe1711575261e37c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:09.334960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229582e0a5f1402fbe1711575261e37c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:09.357673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1108/2229] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:28:08 2022] DELETE /identity/v3/users/229582e0a5f1402fbe1711575261e37c => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:09.363995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe8183-0229-40e2-ac95-efca2ce07c90 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:09.364832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe8183-0229-40e2-ac95-efca2ce07c90 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:09.364832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe8183-0229-40e2-ac95-efca2ce07c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:09.395912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3ffe8183-0229-40e2-ac95-efca2ce07c90 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:09.397173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3ffe8183-0229-40e2-ac95-efca2ce07c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:09.504869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1122/2230] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:09.511727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:09.515467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:09.610717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d4440-23a7-4a27-b917-7eb1f60969fe admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:09.614462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5f413-8952-4ac9-84c2-6b5bcf28576c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:09.615123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5f413-8952-4ac9-84c2-6b5bcf28576c service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:09.615414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5f413-8952-4ac9-84c2-6b5bcf28576c service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:09.615695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5f413-8952-4ac9-84c2-6b5bcf28576c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:09.636971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5f413-8952-4ac9-84c2-6b5bcf28576c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:09.639449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5f413-8952-4ac9-84c2-6b5bcf28576c service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:09.659826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1109/2231] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:28:10.814767 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe8183-0229-40e2-ac95-efca2ce07c90 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:10.816409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3ffe8183-0229-40e2-ac95-efca2ce07c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:10.828596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe8183-0229-40e2-ac95-efca2ce07c90 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:10.833762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:10.835264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:10.835264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:10.835264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] PATH_INFO: `/v3/projects/f8fd6fab9a664be998ad9ba19a9a34a0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:10.837231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8fd6fab9a664be998ad9ba19a9a34a0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:10.840014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:11.348821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:11.372796 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8fd6fab9a664be998ad9ba19a9a34a0', 'request_id': 'req-4324cb30-4908-441e-8daa-f0e35eede0e0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:11.382400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8fd6fab9a664be998ad9ba19a9a34a0', 'request_id': 'req-4324cb30-4908-441e-8daa-f0e35eede0e0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:11.489668 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8fd6fab9a664be998ad9ba19a9a34a0', 'request_id': 'req-4324cb30-4908-441e-8daa-f0e35eede0e0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:11.514159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] The token cache is being invalidate because project f8fd6fab9a664be998ad9ba19a9a34a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:28:11.514813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:11.515909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4324cb30-4908-441e-8daa-f0e35eede0e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:11.517928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1123/2232] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:28:10 2022] DELETE /identity/v3/projects/f8fd6fab9a664be998ad9ba19a9a34a0 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:11.527228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c4b13-d0f8-4c20-92f1-7d853d651d55 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:11.527228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c4b13-d0f8-4c20-92f1-7d853d651d55 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:11.527228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c4b13-d0f8-4c20-92f1-7d853d651d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:11.554299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fe1c4b13-d0f8-4c20-92f1-7d853d651d55 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:11.556968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fe1c4b13-d0f8-4c20-92f1-7d853d651d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:11.594289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:11.661931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1110/2233] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:11.673006 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af5917-e848-41c2-ae16-31f4d14790ad None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:11.673186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af5917-e848-41c2-ae16-31f4d14790ad None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:11.673465 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af5917-e848-41c2-ae16-31f4d14790ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:11.695970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-00af5917-e848-41c2-ae16-31f4d14790ad None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:11.697343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-00af5917-e848-41c2-ae16-31f4d14790ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:11.717703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1124/2234] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:11.719009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c4b13-d0f8-4c20-92f1-7d853d651d55 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:11.739397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c4b13-d0f8-4c20-92f1-7d853d651d55 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:11.756771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8afab-cce4-4469-8d3c-22af2a5aba95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:11.758526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8afab-cce4-4469-8d3c-22af2a5aba95 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:11.759048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8afab-cce4-4469-8d3c-22af2a5aba95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:11.759481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8afab-cce4-4469-8d3c-22af2a5aba95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:11.762757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8afab-cce4-4469-8d3c-22af2a5aba95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:28:11.763133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8afab-cce4-4469-8d3c-22af2a5aba95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:11.765573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8afab-cce4-4469-8d3c-22af2a5aba95 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:11.785224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1111/2235] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:28:11.791305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af5917-e848-41c2-ae16-31f4d14790ad None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:11.801372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af5917-e848-41c2-ae16-31f4d14790ad None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:11.806838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f57c11-8e18-4dbb-bba3-cbf56e2877d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:11.817732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f57c11-8e18-4dbb-bba3-cbf56e2877d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:11.817732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f57c11-8e18-4dbb-bba3-cbf56e2877d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:11.817732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f57c11-8e18-4dbb-bba3-cbf56e2877d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:11.817732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f57c11-8e18-4dbb-bba3-cbf56e2877d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:11.817732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f57c11-8e18-4dbb-bba3-cbf56e2877d2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:11.967030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1125/2236] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:28:11 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:28:11.969943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8afab-cce4-4469-8d3c-22af2a5aba95 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:11.981583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8afab-cce4-4469-8d3c-22af2a5aba95 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:12.057931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61420bd4-5d86-44a9-9ef0-a84d5f8c8680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:12.058652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61420bd4-5d86-44a9-9ef0-a84d5f8c8680 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:12.058943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61420bd4-5d86-44a9-9ef0-a84d5f8c8680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:12.059218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61420bd4-5d86-44a9-9ef0-a84d5f8c8680 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:12.059942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61420bd4-5d86-44a9-9ef0-a84d5f8c8680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:12.061938 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61420bd4-5d86-44a9-9ef0-a84d5f8c8680 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:12.243295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1112/2237] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:28:11 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:28:12.252449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f57c11-8e18-4dbb-bba3-cbf56e2877d2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:12.259463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f57c11-8e18-4dbb-bba3-cbf56e2877d2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:12.265578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55c18b-973d-411f-ad46-473e8c3d2797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:12.266271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55c18b-973d-411f-ad46-473e8c3d2797 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:12.266573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55c18b-973d-411f-ad46-473e8c3d2797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:12.266854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55c18b-973d-411f-ad46-473e8c3d2797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:12.267579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55c18b-973d-411f-ad46-473e8c3d2797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:12.269185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55c18b-973d-411f-ad46-473e8c3d2797 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:12.281914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1126/2238] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:28:12.290554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61420bd4-5d86-44a9-9ef0-a84d5f8c8680 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:12.296872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61420bd4-5d86-44a9-9ef0-a84d5f8c8680 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:12.301508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82996e10-bb69-43d9-b071-7c6081175279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:12.302496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82996e10-bb69-43d9-b071-7c6081175279 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:12.302784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82996e10-bb69-43d9-b071-7c6081175279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:12.303066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82996e10-bb69-43d9-b071-7c6081175279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:12.303820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82996e10-bb69-43d9-b071-7c6081175279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:12.305411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82996e10-bb69-43d9-b071-7c6081175279 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:12.318045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1113/2239] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:28:12.324977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55c18b-973d-411f-ad46-473e8c3d2797 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:12.334353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55c18b-973d-411f-ad46-473e8c3d2797 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:12.340216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389737e-25e0-45f9-a792-314c293126f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:12.341123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389737e-25e0-45f9-a792-314c293126f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:12.341418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389737e-25e0-45f9-a792-314c293126f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:12.341698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389737e-25e0-45f9-a792-314c293126f3 admin admin] PATH_INFO: `/v3/projects/721e2a1b9cbe4c6087a4abdf0427d1c3/users/cfeb0bd06b084db6be61e8b180db060d/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:12.360173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389737e-25e0-45f9-a792-314c293126f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=721e2a1b9cbe4c6087a4abdf0427d1c3, user_id=cfeb0bd06b084db6be61e8b180db060d, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:12.362950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389737e-25e0-45f9-a792-314c293126f3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:12.507480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1127/2240] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:28:12 2022] PUT /identity/v3/projects/721e2a1b9cbe4c6087a4abdf0427d1c3/users/cfeb0bd06b084db6be61e8b180db060d/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:12.519040 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf859c3-0062-4d27-8c44-d6c1d6071da7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:12.519312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf859c3-0062-4d27-8c44-d6c1d6071da7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:12.519600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf859c3-0062-4d27-8c44-d6c1d6071da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:12.557197 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-eaf859c3-0062-4d27-8c44-d6c1d6071da7 None None] MFA Rules not processed for user `cfeb0bd06b084db6be61e8b180db060d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:12.559701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-eaf859c3-0062-4d27-8c44-d6c1d6071da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:12.649574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1114/2241] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:28:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:12.663666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389737e-25e0-45f9-a792-314c293126f3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:12.665448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d389737e-25e0-45f9-a792-314c293126f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:12.742357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389737e-25e0-45f9-a792-314c293126f3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:12.746947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba66425-bef3-4b04-81e2-1c8db7186dd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:12.747606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba66425-bef3-4b04-81e2-1c8db7186dd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:12.747886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba66425-bef3-4b04-81e2-1c8db7186dd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:12.748243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba66425-bef3-4b04-81e2-1c8db7186dd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:12.757932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba66425-bef3-4b04-81e2-1c8db7186dd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:12.759415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba66425-bef3-4b04-81e2-1c8db7186dd4 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:12.773315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1128/2242] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:28:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:20.280528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbb692-e793-4500-bc9d-76390b67e4c5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:20.281086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbb692-e793-4500-bc9d-76390b67e4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:20.281308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbb692-e793-4500-bc9d-76390b67e4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:20.301264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7fcbb692-e793-4500-bc9d-76390b67e4c5 None None] MFA Rules not processed for user `f0a98f1c068843a684f6765d41cc4e49`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:20.302800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7fcbb692-e793-4500-bc9d-76390b67e4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:20.323433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:20.371803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1115/2243] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:28:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:22.766480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:26.653025 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e1f4-4d99-4257-94eb-adc4ca3f7f4c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:26.654425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e1f4-4d99-4257-94eb-adc4ca3f7f4c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:26.654741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286e1f4-4d99-4257-94eb-adc4ca3f7f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:26.677457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6286e1f4-4d99-4257-94eb-adc4ca3f7f4c None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:26.678644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6286e1f4-4d99-4257-94eb-adc4ca3f7f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:26.752137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1129/2244] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:28:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:26.802245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbb692-e793-4500-bc9d-76390b67e4c5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:26.805641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7fcbb692-e793-4500-bc9d-76390b67e4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:26.891535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbb692-e793-4500-bc9d-76390b67e4c5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:26.897759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5123e6-e39e-4067-9bd5-5cfe505ef2af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:26.898522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5123e6-e39e-4067-9bd5-5cfe505ef2af service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:26.898816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5123e6-e39e-4067-9bd5-5cfe505ef2af service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:26.899139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5123e6-e39e-4067-9bd5-5cfe505ef2af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:26.914846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5123e6-e39e-4067-9bd5-5cfe505ef2af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:26.921882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5123e6-e39e-4067-9bd5-5cfe505ef2af service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:26.939454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1116/2245] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:28:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:28:29.179096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e1f4-4d99-4257-94eb-adc4ca3f7f4c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:29.284733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286e1f4-4d99-4257-94eb-adc4ca3f7f4c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:29.294985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:29.296915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:29.297755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:29.299561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] PATH_INFO: `/v3/users/cfeb0bd06b084db6be61e8b180db060d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:29.303449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfeb0bd06b084db6be61e8b180db060d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:29.307859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:29.602487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfeb0bd06b084db6be61e8b180db060d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:29.625371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfeb0bd06b084db6be61e8b180db060d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:29.626742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfeb0bd06b084db6be61e8b180db060d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:29.628522 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfeb0bd06b084db6be61e8b180db060d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:29.751608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfeb0bd06b084db6be61e8b180db060d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:29.781548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1130/2246] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:28:29 2022] DELETE /identity/v3/users/cfeb0bd06b084db6be61e8b180db060d => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:29.782539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8155f5-8209-4a8a-b8f8-5e7f86447388 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:29.783056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8155f5-8209-4a8a-b8f8-5e7f86447388 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:29.783255 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8155f5-8209-4a8a-b8f8-5e7f86447388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:29.807160 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7c8155f5-8209-4a8a-b8f8-5e7f86447388 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:29.808564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7c8155f5-8209-4a8a-b8f8-5e7f86447388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:29.893759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1117/2247] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:28:29.910313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:29.912274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:29.978096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:30.029149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c37eed-b0f6-42fa-b0a1-7c1a5888cf33 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:30.033762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fe1b0-fbaa-4a4c-985d-3cc5fe23e7e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:30.034755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe1b0-fbaa-4a4c-985d-3cc5fe23e7e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:30.035144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe1b0-fbaa-4a4c-985d-3cc5fe23e7e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:30.035552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe1b0-fbaa-4a4c-985d-3cc5fe23e7e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:30.054104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fe1b0-fbaa-4a4c-985d-3cc5fe23e7e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:30.055892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fe1b0-fbaa-4a4c-985d-3cc5fe23e7e1 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:30.074768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1131/2248] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:31.147402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8155f5-8209-4a8a-b8f8-5e7f86447388 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:31.147402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7c8155f5-8209-4a8a-b8f8-5e7f86447388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:31.162124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8155f5-8209-4a8a-b8f8-5e7f86447388 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:31.170616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:31.172195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:31.172490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:31.172803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] PATH_INFO: `/v3/projects/721e2a1b9cbe4c6087a4abdf0427d1c3` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:31.188692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=721e2a1b9cbe4c6087a4abdf0427d1c3)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:31.188692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:31.350081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:31.352247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '721e2a1b9cbe4c6087a4abdf0427d1c3', 'request_id': 'req-28dd86ac-418a-4ab3-b079-5f72da5278fe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:31.361784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '721e2a1b9cbe4c6087a4abdf0427d1c3', 'request_id': 'req-28dd86ac-418a-4ab3-b079-5f72da5278fe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:31.464220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '721e2a1b9cbe4c6087a4abdf0427d1c3', 'request_id': 'req-28dd86ac-418a-4ab3-b079-5f72da5278fe'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:31.486589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] The token cache is being invalidate because project 721e2a1b9cbe4c6087a4abdf0427d1c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:28:31.486976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:31.487986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-28dd86ac-418a-4ab3-b079-5f72da5278fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:31.491282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1118/2249] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:28:31 2022] DELETE /identity/v3/projects/721e2a1b9cbe4c6087a4abdf0427d1c3 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:28:32.450904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fe1b0-fbaa-4a4c-985d-3cc5fe23e7e1 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:32.452442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-633fe1b0-fbaa-4a4c-985d-3cc5fe23e7e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:32.467218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fe1b0-fbaa-4a4c-985d-3cc5fe23e7e1 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:32.532938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:32.533606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:32.533893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:32.534171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:32.536712 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:32.620569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:32.622331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:32.637348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1132/2250] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:28:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:38.610313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba63603-e17f-4e0c-82c3-8ac5ed43bb1a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:38.611947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba63603-e17f-4e0c-82c3-8ac5ed43bb1a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:38.612471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba63603-e17f-4e0c-82c3-8ac5ed43bb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:38.642026 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7ba63603-e17f-4e0c-82c3-8ac5ed43bb1a None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:38.644653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7ba63603-e17f-4e0c-82c3-8ac5ed43bb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:38.798890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1119/2251] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:28:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:38.821449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:38.823903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:38.922552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a87ac-0ed7-4428-8e36-1a742b59f3ba service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:38.929831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a0289-3638-400f-b2b5-293a212d0304 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:38.930557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0289-3638-400f-b2b5-293a212d0304 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:38.932415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0289-3638-400f-b2b5-293a212d0304 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:38.932415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a0289-3638-400f-b2b5-293a212d0304 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:38.941724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a0289-3638-400f-b2b5-293a212d0304 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:38.943761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a0289-3638-400f-b2b5-293a212d0304 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:38.957929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1133/2252] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:28:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:28:41.513357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65266-ba29-413f-bf19-5f33c3e7a3ac None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:41.513930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65266-ba29-413f-bf19-5f33c3e7a3ac None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:41.515640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af65266-ba29-413f-bf19-5f33c3e7a3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:41.534627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6af65266-ba29-413f-bf19-5f33c3e7a3ac None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:41.536338 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6af65266-ba29-413f-bf19-5f33c3e7a3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:41.562734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1120/2253] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 19 12:28:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:41.581326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a0289-3638-400f-b2b5-293a212d0304 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:41.594736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a0289-3638-400f-b2b5-293a212d0304 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:41.606574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c271f-549b-41a8-b6c3-84d7624894a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:41.607436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c271f-549b-41a8-b6c3-84d7624894a9 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:41.607917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c271f-549b-41a8-b6c3-84d7624894a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:41.608563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c271f-549b-41a8-b6c3-84d7624894a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:41.617456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:41.630204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c271f-549b-41a8-b6c3-84d7624894a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:41.634072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c271f-549b-41a8-b6c3-84d7624894a9 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:41.671506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1134/2254] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:28:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:28:42.649700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13e286-279a-474a-b09b-ff9bb8ef0da1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:42.653090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13e286-279a-474a-b09b-ff9bb8ef0da1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:42.653090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13e286-279a-474a-b09b-ff9bb8ef0da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:42.716703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1c13e286-279a-474a-b09b-ff9bb8ef0da1 None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:42.718262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1c13e286-279a-474a-b09b-ff9bb8ef0da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:42.729037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1121/2255] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:28:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:42.747573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c271f-549b-41a8-b6c3-84d7624894a9 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:42.774149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c271f-549b-41a8-b6c3-84d7624894a9 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:42.782021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ad04a-ad5d-4ad2-ab6a-3da110b55eb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:42.782832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ad04a-ad5d-4ad2-ab6a-3da110b55eb7 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:42.783351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ad04a-ad5d-4ad2-ab6a-3da110b55eb7 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:42.783854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ad04a-ad5d-4ad2-ab6a-3da110b55eb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:42.797959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ad04a-ad5d-4ad2-ab6a-3da110b55eb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:42.799756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ad04a-ad5d-4ad2-ab6a-3da110b55eb7 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:42.814474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1135/2256] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:28:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:28:43.675108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e4dd-3103-4a4e-9762-204e85601b27 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:43.675577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e4dd-3103-4a4e-9762-204e85601b27 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:43.675748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658e4dd-3103-4a4e-9762-204e85601b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:43.699416 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a658e4dd-3103-4a4e-9762-204e85601b27 None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:43.700192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a658e4dd-3103-4a4e-9762-204e85601b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:43.708599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1122/2257] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 19 12:28:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:43.722290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ad04a-ad5d-4ad2-ab6a-3da110b55eb7 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:43.734071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ad04a-ad5d-4ad2-ab6a-3da110b55eb7 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:43.739927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4b673-edee-44ec-819a-85879a8c08fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:43.741095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4b673-edee-44ec-819a-85879a8c08fd service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:43.741382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4b673-edee-44ec-819a-85879a8c08fd service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:43.741662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4b673-edee-44ec-819a-85879a8c08fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:43.751386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4b673-edee-44ec-819a-85879a8c08fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:43.753667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4b673-edee-44ec-819a-85879a8c08fd service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:43.763293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1136/2258] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:28:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:28:46.799380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658e4dd-3103-4a4e-9762-204e85601b27 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:46.800924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a658e4dd-3103-4a4e-9762-204e85601b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:46.866645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658e4dd-3103-4a4e-9762-204e85601b27 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:46.871204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:46.872115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:46.872399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:46.872675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] PATH_INFO: `/v3/users/cc67af013a0c4299acd7f2b4de73f7fa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:46.873887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc67af013a0c4299acd7f2b4de73f7fa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:46.876080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:47.144105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc67af013a0c4299acd7f2b4de73f7fa'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:47.145211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc67af013a0c4299acd7f2b4de73f7fa'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:47.162967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:47.164186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc67af013a0c4299acd7f2b4de73f7fa'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:47.165051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc67af013a0c4299acd7f2b4de73f7fa'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:47.269085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc67af013a0c4299acd7f2b4de73f7fa'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:47.298816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1123/2259] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:28:46 2022] DELETE /identity/v3/users/cc67af013a0c4299acd7f2b4de73f7fa => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:47.304856 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f60429-7eb2-4fc3-a0e6-006071a0d13f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:47.305399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f60429-7eb2-4fc3-a0e6-006071a0d13f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:47.305607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f60429-7eb2-4fc3-a0e6-006071a0d13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:47.330105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-26f60429-7eb2-4fc3-a0e6-006071a0d13f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:47.331395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-26f60429-7eb2-4fc3-a0e6-006071a0d13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:47.436408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1137/2260] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:47.449655 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:47.452985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:47.546467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b654c-d0f4-4df0-8df6-673cbeaf7c4c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:47.553468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7dc7d-0b51-428b-abac-f6283fd73833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:47.554988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7dc7d-0b51-428b-abac-f6283fd73833 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:47.555554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7dc7d-0b51-428b-abac-f6283fd73833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:47.556067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7dc7d-0b51-428b-abac-f6283fd73833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:47.575069 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7dc7d-0b51-428b-abac-f6283fd73833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:47.577101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7dc7d-0b51-428b-abac-f6283fd73833 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:47.589409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1124/2261] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:47.899328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f60429-7eb2-4fc3-a0e6-006071a0d13f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:47.901319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-26f60429-7eb2-4fc3-a0e6-006071a0d13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:47.924785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f60429-7eb2-4fc3-a0e6-006071a0d13f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:47.931630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:47.933132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:47.933451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:47.933692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] PATH_INFO: `/v3/projects/566bbc906fb54ffab4964acccd8e68d8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:47.935210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=566bbc906fb54ffab4964acccd8e68d8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:47.937277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:48.090276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '566bbc906fb54ffab4964acccd8e68d8', 'request_id': 'req-8c39253f-2180-4086-8920-f86382066bd4'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:48.096124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '566bbc906fb54ffab4964acccd8e68d8', 'request_id': 'req-8c39253f-2180-4086-8920-f86382066bd4'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:48.185363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '566bbc906fb54ffab4964acccd8e68d8', 'request_id': 'req-8c39253f-2180-4086-8920-f86382066bd4'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:48.197387 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] The token cache is being invalidate because project 566bbc906fb54ffab4964acccd8e68d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:28:48.197728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:48.198363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-8c39253f-2180-4086-8920-f86382066bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:28:48.199923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1138/2262] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:28:47 2022] DELETE /identity/v3/projects/566bbc906fb54ffab4964acccd8e68d8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:28:50.689446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a02da-812c-44d6-90d7-15892346f71e None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:50.689995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a02da-812c-44d6-90d7-15892346f71e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:50.689995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a02da-812c-44d6-90d7-15892346f71e None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:50.691436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1125/2263] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 12:28:50.698034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b04e7-77f9-4876-91bb-bcb88e09bfa8 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:50.698639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b04e7-77f9-4876-91bb-bcb88e09bfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:50.698723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b04e7-77f9-4876-91bb-bcb88e09bfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:50.742843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ef9b04e7-77f9-4876-91bb-bcb88e09bfa8 None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:50.744110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ef9b04e7-77f9-4876-91bb-bcb88e09bfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:50.787986 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:50.827512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1139/2264] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:28:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:50.839924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a02da-812c-44d6-90d7-15892346f71e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:50.850351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6c7a02da-812c-44d6-90d7-15892346f71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:50.923128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a02da-812c-44d6-90d7-15892346f71e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:50.927061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a92cc9-b986-4b80-814c-df10c776a8bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9969632e796345fa9671a45cf61ede3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:50.927967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a92cc9-b986-4b80-814c-df10c776a8bd service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:50.928248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a92cc9-b986-4b80-814c-df10c776a8bd service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:50.928525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a92cc9-b986-4b80-814c-df10c776a8bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:50.939815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a92cc9-b986-4b80-814c-df10c776a8bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:50.941278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a92cc9-b986-4b80-814c-df10c776a8bd service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:50.953759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1126/2265] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:28:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:51.215137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80769a-8936-4381-9d39-0565d7e50604 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:51.215600 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80769a-8936-4381-9d39-0565d7e50604 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:51.215883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80769a-8936-4381-9d39-0565d7e50604 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:51.217294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1140/2266] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:28:51.223411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844ea1d-323d-437d-bab1-be16ecf3ac0c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:51.223883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844ea1d-323d-437d-bab1-be16ecf3ac0c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:51.224330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844ea1d-323d-437d-bab1-be16ecf3ac0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:51.244983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8844ea1d-323d-437d-bab1-be16ecf3ac0c None None] MFA Rules not processed for user `8b266ea9964c4a179c2743b9cf6478eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:51.246206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8844ea1d-323d-437d-bab1-be16ecf3ac0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:51.302771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1127/2267] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:28:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:51.316438 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80769a-8936-4381-9d39-0565d7e50604 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:51.318342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bd80769a-8936-4381-9d39-0565d7e50604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:51.332490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80769a-8936-4381-9d39-0565d7e50604 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:51.336036 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a68345-1c94-478c-8e61-6e282b34bd2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:51.336678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a68345-1c94-478c-8e61-6e282b34bd2c service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:51.336964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a68345-1c94-478c-8e61-6e282b34bd2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:51.337241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a68345-1c94-478c-8e61-6e282b34bd2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:51.344702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a68345-1c94-478c-8e61-6e282b34bd2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:51.346129 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a68345-1c94-478c-8e61-6e282b34bd2c service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:51.354026 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1141/2268] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:28:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:28:51.443260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3c58d-6107-4f22-a273-6b8aedc8a0cc None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:51.443548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3c58d-6107-4f22-a273-6b8aedc8a0cc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:51.443830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3c58d-6107-4f22-a273-6b8aedc8a0cc None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:51.445220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1128/2269] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:28:51.451685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41038fc2-c805-4e67-b581-0f7c314dfe62 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:51.452021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41038fc2-c805-4e67-b581-0f7c314dfe62 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:51.452330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41038fc2-c805-4e67-b581-0f7c314dfe62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:51.471192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-41038fc2-c805-4e67-b581-0f7c314dfe62 None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:51.472418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-41038fc2-c805-4e67-b581-0f7c314dfe62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:51.482101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1142/2270] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:28:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:51.493095 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3c58d-6107-4f22-a273-6b8aedc8a0cc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:51.505608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3c58d-6107-4f22-a273-6b8aedc8a0cc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:51.509261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92eb2a-f76d-4525-b14e-87d56f7ab931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:51.510688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92eb2a-f76d-4525-b14e-87d56f7ab931 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:51.511016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92eb2a-f76d-4525-b14e-87d56f7ab931 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:51.511364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92eb2a-f76d-4525-b14e-87d56f7ab931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:51.522395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92eb2a-f76d-4525-b14e-87d56f7ab931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:51.523881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92eb2a-f76d-4525-b14e-87d56f7ab931 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:51.535750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1129/2271] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:28:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:28:55.737213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72251788-1b80-42ba-90d2-f7ef5bc0b7af None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:55.738002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72251788-1b80-42ba-90d2-f7ef5bc0b7af None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:55.738002 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72251788-1b80-42ba-90d2-f7ef5bc0b7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:55.757638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-72251788-1b80-42ba-90d2-f7ef5bc0b7af None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:28:55.758947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-72251788-1b80-42ba-90d2-f7ef5bc0b7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:55.818511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1143/2272] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:28:55.833770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92eb2a-f76d-4525-b14e-87d56f7ab931 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:28:55.835490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-dc92eb2a-f76d-4525-b14e-87d56f7ab931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:28:55.867353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:28:55.917244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92eb2a-f76d-4525-b14e-87d56f7ab931 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:28:55.931062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ed263-4d92-418a-bec4-e3b0e5bf7300 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:28:55.931794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ed263-4d92-418a-bec4-e3b0e5bf7300 service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:28:55.932067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ed263-4d92-418a-bec4-e3b0e5bf7300 service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:28:55.932357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ed263-4d92-418a-bec4-e3b0e5bf7300 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:28:55.942153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ed263-4d92-418a-bec4-e3b0e5bf7300 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:28:55.943882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ed263-4d92-418a-bec4-e3b0e5bf7300 service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:28:55.953067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1130/2273] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:28:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 12:29:05.811793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=48 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:06.079097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=22 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:16.087661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72251788-1b80-42ba-90d2-f7ef5bc0b7af None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:16.097248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72251788-1b80-42ba-90d2-f7ef5bc0b7af None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:16.101111 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575ad67-9de8-4202-848d-521b1daa8347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:16.101759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575ad67-9de8-4202-848d-521b1daa8347 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:16.102041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575ad67-9de8-4202-848d-521b1daa8347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:16.102316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575ad67-9de8-4202-848d-521b1daa8347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:16.104192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6575ad67-9de8-4202-848d-521b1daa8347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:16.118491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575ad67-9de8-4202-848d-521b1daa8347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:16.120408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575ad67-9de8-4202-848d-521b1daa8347 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:16.132316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1144/2274] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:29:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:17.322285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ed263-4d92-418a-bec4-e3b0e5bf7300 service swift] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:17.333404 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ed263-4d92-418a-bec4-e3b0e5bf7300 service swift] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:17.337908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ed8b-a420-4d60-a293-a6b683585499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:17.339700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ed8b-a420-4d60-a293-a6b683585499 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:17.340195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ed8b-a420-4d60-a293-a6b683585499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:17.340482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168ed8b-a420-4d60-a293-a6b683585499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:17.343044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c168ed8b-a420-4d60-a293-a6b683585499 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:17.359520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168ed8b-a420-4d60-a293-a6b683585499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:17.361209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168ed8b-a420-4d60-a293-a6b683585499 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:17.379714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1131/2275] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:29:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:26.127133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=6 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:27.198740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ddf75-1c8d-4d82-9afc-fb2e403d49e3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:27.199416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ddf75-1c8d-4d82-9afc-fb2e403d49e3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:27.199416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ddf75-1c8d-4d82-9afc-fb2e403d49e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:27.218390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f95ddf75-1c8d-4d82-9afc-fb2e403d49e3 None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:29:27.219647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f95ddf75-1c8d-4d82-9afc-fb2e403d49e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:27.227330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1145/2276] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 19 12:29:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:29:27.251382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ed8b-a420-4d60-a293-a6b683585499 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:27.264034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168ed8b-a420-4d60-a293-a6b683585499 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:27.268993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0401bb9-1acd-4269-9b86-92567717e165 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:27.269732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0401bb9-1acd-4269-9b86-92567717e165 service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:27.270250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0401bb9-1acd-4269-9b86-92567717e165 service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:27.270570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0401bb9-1acd-4269-9b86-92567717e165 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:27.279947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0401bb9-1acd-4269-9b86-92567717e165 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:27.281630 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0401bb9-1acd-4269-9b86-92567717e165 service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:27.290185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1132/2277] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:29:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:29:28.069533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ddf75-1c8d-4d82-9afc-fb2e403d49e3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:28.070935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f95ddf75-1c8d-4d82-9afc-fb2e403d49e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:28.087549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ddf75-1c8d-4d82-9afc-fb2e403d49e3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:28.094057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6cefe-7d4b-43bf-8a9b-e10bb1f2cd58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:28.094765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6cefe-7d4b-43bf-8a9b-e10bb1f2cd58 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:28.095252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6cefe-7d4b-43bf-8a9b-e10bb1f2cd58 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:28.095425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6cefe-7d4b-43bf-8a9b-e10bb1f2cd58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:28.097347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8aa6cefe-7d4b-43bf-8a9b-e10bb1f2cd58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:28.180295 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6cefe-7d4b-43bf-8a9b-e10bb1f2cd58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:28.181893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6cefe-7d4b-43bf-8a9b-e10bb1f2cd58 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:28.192733 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1146/2278] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:37.286901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=12 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:38.187709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=31 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:41.961428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0401bb9-1acd-4269-9b86-92567717e165 service swift] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:41.963377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c0401bb9-1acd-4269-9b86-92567717e165 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:41.975901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0401bb9-1acd-4269-9b86-92567717e165 service swift] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:41.979811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:41.980452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:41.980737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:41.981014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:41.982967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:41.996849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:42.002156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:42.008304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1133/2279] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:29:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:44.896484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe52ea-edcc-48e0-a6ae-65dd75dfe484 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:44.897044 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe52ea-edcc-48e0-a6ae-65dd75dfe484 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:44.897044 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe52ea-edcc-48e0-a6ae-65dd75dfe484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:44.915184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a2fe52ea-edcc-48e0-a6ae-65dd75dfe484 None None] MFA Rules not processed for user `5c35bd3640694b0dbce13c713fd68306`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:29:44.916409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a2fe52ea-edcc-48e0-a6ae-65dd75dfe484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:44.925381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1147/2280] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:29:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:29:52.004304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:52.894129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:52.896046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:52.914883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcbd06-f376-4373-9b0b-4836d71b8f09 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:52.923795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a9f63-5f3d-4241-8027-43d11133567c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:52.923795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a9f63-5f3d-4241-8027-43d11133567c service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:52.923795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a9f63-5f3d-4241-8027-43d11133567c service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:52.923795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a9f63-5f3d-4241-8027-43d11133567c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:52.924340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-335a9f63-5f3d-4241-8027-43d11133567c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:52.936911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a9f63-5f3d-4241-8027-43d11133567c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:52.938551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a9f63-5f3d-4241-8027-43d11133567c service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:52.949282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1134/2281] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:29:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:29:54.910224 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:29:58.296420 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe52ea-edcc-48e0-a6ae-65dd75dfe484 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:29:58.310552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe52ea-edcc-48e0-a6ae-65dd75dfe484 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:29:58.316755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f5f6c-1a48-4fda-8eac-793f4f679942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:29:58.317411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f5f6c-1a48-4fda-8eac-793f4f679942 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:29:58.317697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f5f6c-1a48-4fda-8eac-793f4f679942 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:29:58.317975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f5f6c-1a48-4fda-8eac-793f4f679942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:29:58.319829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-685f5f6c-1a48-4fda-8eac-793f4f679942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:29:58.381194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f5f6c-1a48-4fda-8eac-793f4f679942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:29:58.382836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f5f6c-1a48-4fda-8eac-793f4f679942 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:29:58.395882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1148/2282] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:29:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:02.944747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:08.388307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=28 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:27.778029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a9f63-5f3d-4241-8027-43d11133567c service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:27.789308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a9f63-5f3d-4241-8027-43d11133567c service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:27.794971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36c6bf-babd-46c7-9c21-3178b1c6212d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:27.797130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36c6bf-babd-46c7-9c21-3178b1c6212d service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:27.797130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36c6bf-babd-46c7-9c21-3178b1c6212d service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:27.797130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36c6bf-babd-46c7-9c21-3178b1c6212d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:27.798652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-da36c6bf-babd-46c7-9c21-3178b1c6212d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:27.869352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36c6bf-babd-46c7-9c21-3178b1c6212d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:27.871019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36c6bf-babd-46c7-9c21-3178b1c6212d service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:27.881163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1135/2283] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:30:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:37.876534 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=28 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:30:46.274984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f5f6c-1a48-4fda-8eac-793f4f679942 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:46.286969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f5f6c-1a48-4fda-8eac-793f4f679942 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:46.293156 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a862e-86fd-4b9d-8f8c-6392d1d70567 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:46.293809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a862e-86fd-4b9d-8f8c-6392d1d70567 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:46.294101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a862e-86fd-4b9d-8f8c-6392d1d70567 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:46.294379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a862e-86fd-4b9d-8f8c-6392d1d70567 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:46.296267 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-845a862e-86fd-4b9d-8f8c-6392d1d70567 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:46.362806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a862e-86fd-4b9d-8f8c-6392d1d70567 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:46.364468 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a862e-86fd-4b9d-8f8c-6392d1d70567 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:46.374590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1149/2284] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:30:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:48.157804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f0dbc-7c98-47fd-8020-1926076ab17f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:48.158734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f0dbc-7c98-47fd-8020-1926076ab17f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:48.159226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f0dbc-7c98-47fd-8020-1926076ab17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:48.179217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-504f0dbc-7c98-47fd-8020-1926076ab17f None None] MFA Rules not processed for user `f0a98f1c068843a684f6765d41cc4e49`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:48.180481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-504f0dbc-7c98-47fd-8020-1926076ab17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:48.187664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1136/2285] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:48.524920 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a862e-86fd-4b9d-8f8c-6392d1d70567 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:48.536359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a862e-86fd-4b9d-8f8c-6392d1d70567 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:48.540039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a05f1c-f6db-4ccb-b0f9-1fe4ab0b2177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:48.540682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a05f1c-f6db-4ccb-b0f9-1fe4ab0b2177 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:48.540962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a05f1c-f6db-4ccb-b0f9-1fe4ab0b2177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:48.541244 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a05f1c-f6db-4ccb-b0f9-1fe4ab0b2177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:48.543054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-48a05f1c-f6db-4ccb-b0f9-1fe4ab0b2177 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:48.554931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a05f1c-f6db-4ccb-b0f9-1fe4ab0b2177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:48.556895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a05f1c-f6db-4ccb-b0f9-1fe4ab0b2177 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:48.568201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1150/2286] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:30:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:50.044934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f0dbc-7c98-47fd-8020-1926076ab17f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:50.046359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-504f0dbc-7c98-47fd-8020-1926076ab17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:50.116400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f0dbc-7c98-47fd-8020-1926076ab17f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:50.121435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d94f9-83e5-49b0-aa63-8e581901b672 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:50.122079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d94f9-83e5-49b0-aa63-8e581901b672 service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:50.122360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d94f9-83e5-49b0-aa63-8e581901b672 service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:50.122696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d94f9-83e5-49b0-aa63-8e581901b672 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:50.124446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-153d94f9-83e5-49b0-aa63-8e581901b672 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:50.136953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d94f9-83e5-49b0-aa63-8e581901b672 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:50.138550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d94f9-83e5-49b0-aa63-8e581901b672 service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:50.149236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1137/2287] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:30:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:30:58.213662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afb47c-a562-4084-8ebb-7fc707c20987 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:58.214204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afb47c-a562-4084-8ebb-7fc707c20987 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:58.214454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afb47c-a562-4084-8ebb-7fc707c20987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:58.234303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f4afb47c-a562-4084-8ebb-7fc707c20987 None None] MFA Rules not processed for user `746a70534fe5419ebe6d5e97a7a041f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:30:58.235579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f4afb47c-a562-4084-8ebb-7fc707c20987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:58.242902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1151/2288] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:30:58.841235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d94f9-83e5-49b0-aa63-8e581901b672 service placement] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:30:58.851859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d94f9-83e5-49b0-aa63-8e581901b672 service placement] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:30:58.855516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1d289-12a3-40fd-a588-1e3763c656c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:30:58.857088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1d289-12a3-40fd-a588-1e3763c656c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:30:58.857709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1d289-12a3-40fd-a588-1e3763c656c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:30:58.858014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1d289-12a3-40fd-a588-1e3763c656c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:30:58.859804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e8d1d289-12a3-40fd-a588-1e3763c656c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:30:58.870242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1d289-12a3-40fd-a588-1e3763c656c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:30:58.871794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1d289-12a3-40fd-a588-1e3763c656c5 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:30:58.880516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1138/2289] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:04.346545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afb47c-a562-4084-8ebb-7fc707c20987 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:04.360657 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afb47c-a562-4084-8ebb-7fc707c20987 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:04.367461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8c407-836d-40b2-a1ee-7a118f85a115 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:04.368110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8c407-836d-40b2-a1ee-7a118f85a115 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:04.368390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8c407-836d-40b2-a1ee-7a118f85a115 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:04.368691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8c407-836d-40b2-a1ee-7a118f85a115 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:04.370566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e8f8c407-836d-40b2-a1ee-7a118f85a115 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:04.384000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8c407-836d-40b2-a1ee-7a118f85a115 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:04.385619 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8c407-836d-40b2-a1ee-7a118f85a115 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:04.397464 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1152/2290] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:31:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:08.508214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1d289-12a3-40fd-a588-1e3763c656c5 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:08.523679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1d289-12a3-40fd-a588-1e3763c656c5 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:08.532041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:08.533382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:08.534013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:08.534630 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:08.537492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:08.552090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:08.554089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:08.569183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1139/2291] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:14.392187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=43 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:18.561086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=47 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:24.988373 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8c407-836d-40b2-a1ee-7a118f85a115 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:24.997821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8c407-836d-40b2-a1ee-7a118f85a115 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:25.001709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0655a-9215-4143-b48f-7b67008c94e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:25.002288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0655a-9215-4143-b48f-7b67008c94e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:25.003049 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0655a-9215-4143-b48f-7b67008c94e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:25.003404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0655a-9215-4143-b48f-7b67008c94e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:25.005325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1cf0655a-9215-4143-b48f-7b67008c94e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:25.074737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0655a-9215-4143-b48f-7b67008c94e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:25.076181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0655a-9215-4143-b48f-7b67008c94e4 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:25.087013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1153/2292] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:31:30.360695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:30.363947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:30.378292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66b3d0-9d6d-45f1-ae3e-de71e328da49 service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:30.383530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:30.384407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:30.384687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:30.384960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] PATH_INFO: `/v3/users/5c35bd3640694b0dbce13c713fd68306` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:30.386280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c35bd3640694b0dbce13c713fd68306)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:30.388289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:30.632048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c35bd3640694b0dbce13c713fd68306'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:30.632925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c35bd3640694b0dbce13c713fd68306'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:30.658584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c35bd3640694b0dbce13c713fd68306'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:30.660029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c35bd3640694b0dbce13c713fd68306'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:30.764970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-41721ca1-8ecc-42da-9007-f8d32e8f3372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c35bd3640694b0dbce13c713fd68306'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:30.780963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1140/2293] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:31:30 2022] DELETE /identity/v3/users/5c35bd3640694b0dbce13c713fd68306 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:31.150055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0655a-9215-4143-b48f-7b67008c94e4 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:31.160074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0655a-9215-4143-b48f-7b67008c94e4 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:31.239105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:31.239982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:31.240264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:31.240600 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] PATH_INFO: `/v3/projects/d9e96ed4a3b5430298605d709d353f6e` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:31.241849 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e96ed4a3b5430298605d709d353f6e)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:31.244052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:31.389400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e96ed4a3b5430298605d709d353f6e', 'request_id': 'req-ac723221-537b-43fa-b030-8206f0ebf1aa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:31.402070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e96ed4a3b5430298605d709d353f6e', 'request_id': 'req-ac723221-537b-43fa-b030-8206f0ebf1aa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:31.518201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e96ed4a3b5430298605d709d353f6e', 'request_id': 'req-ac723221-537b-43fa-b030-8206f0ebf1aa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:31.532273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] The token cache is being invalidate because project d9e96ed4a3b5430298605d709d353f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:31.532720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:31.533391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac723221-537b-43fa-b030-8206f0ebf1aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:31.535633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1154/2294] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:31:31 2022] DELETE /identity/v3/projects/d9e96ed4a3b5430298605d709d353f6e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:31.540960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab0218-902e-4047-a324-748413d77574 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:31.541710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab0218-902e-4047-a324-748413d77574 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:31.541710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab0218-902e-4047-a324-748413d77574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:31.566256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3fab0218-902e-4047-a324-748413d77574 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:31.567509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3fab0218-902e-4047-a324-748413d77574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:31.628885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1141/2295] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:31.636875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70092410-f37e-4d82-af10-b030a240f58c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:31.637163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70092410-f37e-4d82-af10-b030a240f58c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:31.637493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70092410-f37e-4d82-af10-b030a240f58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:31.659325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-70092410-f37e-4d82-af10-b030a240f58c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:31.660901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-70092410-f37e-4d82-af10-b030a240f58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:31.669204 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1155/2296] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:31.674155 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab0218-902e-4047-a324-748413d77574 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:31.685286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab0218-902e-4047-a324-748413d77574 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:31.690859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc87668-636d-4770-b9d5-5533ba01b203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:31.691562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc87668-636d-4770-b9d5-5533ba01b203 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:31.691834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc87668-636d-4770-b9d5-5533ba01b203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:31.692131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc87668-636d-4770-b9d5-5533ba01b203 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:31.692874 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc87668-636d-4770-b9d5-5533ba01b203 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:31:31.693181 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc87668-636d-4770-b9d5-5533ba01b203 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:31.695076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc87668-636d-4770-b9d5-5533ba01b203 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:31.718025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1142/2297] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:31:31.724927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70092410-f37e-4d82-af10-b030a240f58c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:31.736711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70092410-f37e-4d82-af10-b030a240f58c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:31.744088 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d033d17-a8a2-4e54-a1c6-bfcb06d18633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:31.745357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d033d17-a8a2-4e54-a1c6-bfcb06d18633 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:31.745498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d033d17-a8a2-4e54-a1c6-bfcb06d18633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:31.745724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d033d17-a8a2-4e54-a1c6-bfcb06d18633 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:31.746632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d033d17-a8a2-4e54-a1c6-bfcb06d18633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:31.748761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d033d17-a8a2-4e54-a1c6-bfcb06d18633 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:31.768050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:31.891182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1156/2298] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:31:31 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:31.892356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc87668-636d-4770-b9d5-5533ba01b203 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:31.903394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc87668-636d-4770-b9d5-5533ba01b203 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:31.977749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcfcf0-933f-46f9-9ed6-6799453d4100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:31.978641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcfcf0-933f-46f9-9ed6-6799453d4100 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:31.979007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcfcf0-933f-46f9-9ed6-6799453d4100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:31.979364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcfcf0-933f-46f9-9ed6-6799453d4100 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:31.980318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcfcf0-933f-46f9-9ed6-6799453d4100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:31.982449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcfcf0-933f-46f9-9ed6-6799453d4100 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:31.991312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:32.079402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1143/2299] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:31:31 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:31:32.084305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d033d17-a8a2-4e54-a1c6-bfcb06d18633 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:32.102437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d033d17-a8a2-4e54-a1c6-bfcb06d18633 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:32.108417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dba79-e21f-48a7-8439-522fb35472d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:32.109109 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dba79-e21f-48a7-8439-522fb35472d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:32.109442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dba79-e21f-48a7-8439-522fb35472d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:32.109722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dba79-e21f-48a7-8439-522fb35472d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:32.110641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dba79-e21f-48a7-8439-522fb35472d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:32.113289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dba79-e21f-48a7-8439-522fb35472d6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:32.130516 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1157/2300] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:32.131384 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcfcf0-933f-46f9-9ed6-6799453d4100 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:32.146612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcfcf0-933f-46f9-9ed6-6799453d4100 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:32.152310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe4360-8061-4ca9-9f96-3d3f0dbe7725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:32.153057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe4360-8061-4ca9-9f96-3d3f0dbe7725 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:32.153348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe4360-8061-4ca9-9f96-3d3f0dbe7725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:32.153623 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe4360-8061-4ca9-9f96-3d3f0dbe7725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:32.154371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe4360-8061-4ca9-9f96-3d3f0dbe7725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:32.156883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe4360-8061-4ca9-9f96-3d3f0dbe7725 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:32.177949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1144/2301] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:31:32.179584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dba79-e21f-48a7-8439-522fb35472d6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:32.190562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dba79-e21f-48a7-8439-522fb35472d6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:32.196212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4186483-1060-4f27-a542-4aa33204fb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:32.197375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4186483-1060-4f27-a542-4aa33204fb88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:32.197823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4186483-1060-4f27-a542-4aa33204fb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:32.198283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4186483-1060-4f27-a542-4aa33204fb88 admin admin] PATH_INFO: `/v3/projects/5d5c453a2fa04819a3cd976ab59eaf0c/users/77104290fa9b4a97850ca238d3df612b/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:32.216979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4186483-1060-4f27-a542-4aa33204fb88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5c453a2fa04819a3cd976ab59eaf0c, user_id=77104290fa9b4a97850ca238d3df612b, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:32.219363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4186483-1060-4f27-a542-4aa33204fb88 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:32.305445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1158/2302] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:31:32 2022] PUT /identity/v3/projects/5d5c453a2fa04819a3cd976ab59eaf0c/users/77104290fa9b4a97850ca238d3df612b/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:32.307833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf5de3-2e20-4278-ab72-fca8ea6bda10 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:32.308186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf5de3-2e20-4278-ab72-fca8ea6bda10 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:32.308888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf5de3-2e20-4278-ab72-fca8ea6bda10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:32.340418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-98cf5de3-2e20-4278-ab72-fca8ea6bda10 None None] MFA Rules not processed for user `77104290fa9b4a97850ca238d3df612b`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:32.344533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-98cf5de3-2e20-4278-ab72-fca8ea6bda10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:32.434660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1145/2303] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:31:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:32.439803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4186483-1060-4f27-a542-4aa33204fb88 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:32.520259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4186483-1060-4f27-a542-4aa33204fb88 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:32.526003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfea7bd-8273-4227-8d51-d1aacba82184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:32.526708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfea7bd-8273-4227-8d51-d1aacba82184 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:32.527160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfea7bd-8273-4227-8d51-d1aacba82184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:32.527623 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfea7bd-8273-4227-8d51-d1aacba82184 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:32.528643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfea7bd-8273-4227-8d51-d1aacba82184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:32.531121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfea7bd-8273-4227-8d51-d1aacba82184 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:32.636779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1159/2304] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:31:32 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:32.641639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf5de3-2e20-4278-ab72-fca8ea6bda10 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:32.654418 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf5de3-2e20-4278-ab72-fca8ea6bda10 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:32.725573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3e80c-d96f-4830-b095-507e8225ad89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:32.726172 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3e80c-d96f-4830-b095-507e8225ad89 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:32.726494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3e80c-d96f-4830-b095-507e8225ad89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:32.726784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3e80c-d96f-4830-b095-507e8225ad89 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:32.727492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3e80c-d96f-4830-b095-507e8225ad89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:32.729585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3e80c-d96f-4830-b095-507e8225ad89 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:32.875078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1146/2305] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:31:32 2022] POST /identity/v3/users => generated 294 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:31:32.876555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfea7bd-8273-4227-8d51-d1aacba82184 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:32.888634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfea7bd-8273-4227-8d51-d1aacba82184 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:32.894931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70776946-0e59-456d-a1f1-a6a94cf0bc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:32.895842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70776946-0e59-456d-a1f1-a6a94cf0bc3a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:32.896135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70776946-0e59-456d-a1f1-a6a94cf0bc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:32.896607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70776946-0e59-456d-a1f1-a6a94cf0bc3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:32.897360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70776946-0e59-456d-a1f1-a6a94cf0bc3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:32.899166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70776946-0e59-456d-a1f1-a6a94cf0bc3a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:32.915452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1160/2306] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:31:32.918945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3e80c-d96f-4830-b095-507e8225ad89 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:32.936755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3e80c-d96f-4830-b095-507e8225ad89 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:32.945607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e49be-a5ab-4b02-9f93-fa2f348a50c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:32.946663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e49be-a5ab-4b02-9f93-fa2f348a50c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:32.947977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e49be-a5ab-4b02-9f93-fa2f348a50c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:32.948347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e49be-a5ab-4b02-9f93-fa2f348a50c6 admin admin] PATH_INFO: `/v3/projects/d8972e15a1934b1fb8ce7557f28de5f1/users/f8dedf6c411548c895bbf323d07ee6a0/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:32.973456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e49be-a5ab-4b02-9f93-fa2f348a50c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8972e15a1934b1fb8ce7557f28de5f1, user_id=f8dedf6c411548c895bbf323d07ee6a0, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:32.977163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e49be-a5ab-4b02-9f93-fa2f348a50c6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:33.040821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1147/2307] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:31:32 2022] PUT /identity/v3/projects/d8972e15a1934b1fb8ce7557f28de5f1/users/f8dedf6c411548c895bbf323d07ee6a0/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:33.045285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70776946-0e59-456d-a1f1-a6a94cf0bc3a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:33.056800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70776946-0e59-456d-a1f1-a6a94cf0bc3a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:33.130343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52249db1-ac0e-4eec-9268-5000a16e04ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:33.131181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52249db1-ac0e-4eec-9268-5000a16e04ae admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:33.131417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52249db1-ac0e-4eec-9268-5000a16e04ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:33.131703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52249db1-ac0e-4eec-9268-5000a16e04ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:33.132549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52249db1-ac0e-4eec-9268-5000a16e04ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:33.134124 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52249db1-ac0e-4eec-9268-5000a16e04ae admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:33.145842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1161/2308] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:31:33.149888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e49be-a5ab-4b02-9f93-fa2f348a50c6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:33.161070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e49be-a5ab-4b02-9f93-fa2f348a50c6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:33.166021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575bd07-bd2f-4fc4-9000-951e5db1737c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:33.166964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575bd07-bd2f-4fc4-9000-951e5db1737c admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:33.167941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575bd07-bd2f-4fc4-9000-951e5db1737c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:33.168238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575bd07-bd2f-4fc4-9000-951e5db1737c admin admin] PATH_INFO: `/v3/projects/d8972e15a1934b1fb8ce7557f28de5f1/users/f8dedf6c411548c895bbf323d07ee6a0/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:33.170795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575bd07-bd2f-4fc4-9000-951e5db1737c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8972e15a1934b1fb8ce7557f28de5f1, user_id=f8dedf6c411548c895bbf323d07ee6a0, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:33.173202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575bd07-bd2f-4fc4-9000-951e5db1737c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:33.185538 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1148/2309] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:31:33 2022] PUT /identity/v3/projects/d8972e15a1934b1fb8ce7557f28de5f1/users/f8dedf6c411548c895bbf323d07ee6a0/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:33.198185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cad7c1-080f-43a9-9dc0-b21decd4bc29 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:33.198727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cad7c1-080f-43a9-9dc0-b21decd4bc29 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:33.198825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cad7c1-080f-43a9-9dc0-b21decd4bc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:33.227286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-89cad7c1-080f-43a9-9dc0-b21decd4bc29 None None] MFA Rules not processed for user `f8dedf6c411548c895bbf323d07ee6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:33.228411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-89cad7c1-080f-43a9-9dc0-b21decd4bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:33.240821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:33.310104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1162/2310] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:33.319829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575bd07-bd2f-4fc4-9000-951e5db1737c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:33.321541 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f575bd07-bd2f-4fc4-9000-951e5db1737c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:33.347643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:33.387673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575bd07-bd2f-4fc4-9000-951e5db1737c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:33.391498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beb575-ec28-4156-8232-2927e55f948a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:33.392261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beb575-ec28-4156-8232-2927e55f948a service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:33.392551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beb575-ec28-4156-8232-2927e55f948a service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:33.392832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85beb575-ec28-4156-8232-2927e55f948a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:33.454494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85beb575-ec28-4156-8232-2927e55f948a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:33.457131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85beb575-ec28-4156-8232-2927e55f948a service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:33.467854 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1149/2311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:31:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:39.075880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523687c-3048-435b-8f9c-60a81a6b31ba None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:39.076479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523687c-3048-435b-8f9c-60a81a6b31ba None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:39.076479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523687c-3048-435b-8f9c-60a81a6b31ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:39.094273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d523687c-3048-435b-8f9c-60a81a6b31ba None None] MFA Rules not processed for user `707a899d6c5d4c4ea1c58e0de6a85b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:39.095628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d523687c-3048-435b-8f9c-60a81a6b31ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:39.156294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1163/2312] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:41.072457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beb575-ec28-4156-8232-2927e55f948a service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:41.090583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85beb575-ec28-4156-8232-2927e55f948a service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:41.090583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f245f-f147-4c92-9f40-77bf460e71af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:41.090583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f245f-f147-4c92-9f40-77bf460e71af service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:41.090583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f245f-f147-4c92-9f40-77bf460e71af service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:41.091061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f245f-f147-4c92-9f40-77bf460e71af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:41.098614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f245f-f147-4c92-9f40-77bf460e71af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:41.100134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f245f-f147-4c92-9f40-77bf460e71af service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:41.110883 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1150/2313] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:31:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:41.124360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2500b-931f-40e6-a983-2324812377be None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:41.124867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2500b-931f-40e6-a983-2324812377be None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:41.124985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2500b-931f-40e6-a983-2324812377be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:41.144113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6eb2500b-931f-40e6-a983-2324812377be None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:31:41.146201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6eb2500b-931f-40e6-a983-2324812377be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:41.208952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1164/2314] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 19 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:31:41.325196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f245f-f147-4c92-9f40-77bf460e71af service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:41.327142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c89f245f-f147-4c92-9f40-77bf460e71af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:41.420149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f245f-f147-4c92-9f40-77bf460e71af service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:41.449432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f852d-9f99-4e11-84e2-295343c8e79a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:41.449432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f852d-9f99-4e11-84e2-295343c8e79a service swift] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:41.449432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f852d-9f99-4e11-84e2-295343c8e79a service swift] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:41.449432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f852d-9f99-4e11-84e2-295343c8e79a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:41.449950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f852d-9f99-4e11-84e2-295343c8e79a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:41.449950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f852d-9f99-4e11-84e2-295343c8e79a service swift] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:41.454724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1151/2315] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:31:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:31:49.008152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2500b-931f-40e6-a983-2324812377be None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:49.009824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6eb2500b-931f-40e6-a983-2324812377be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:49.079748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2500b-931f-40e6-a983-2324812377be None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:49.084330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6c497-0697-47b1-9f48-f350a4bba4e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:49.085003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6c497-0697-47b1-9f48-f350a4bba4e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:49.085296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6c497-0697-47b1-9f48-f350a4bba4e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:49.085582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6c497-0697-47b1-9f48-f350a4bba4e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:49.087486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-83e6c497-0697-47b1-9f48-f350a4bba4e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:49.100842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:49.158010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6c497-0697-47b1-9f48-f350a4bba4e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:49.160101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6c497-0697-47b1-9f48-f350a4bba4e2 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:49.170548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1165/2316] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:31:51.446073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=77 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:51.448830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f852d-9f99-4e11-84e2-295343c8e79a service swift] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:51.450352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4c6f852d-9f99-4e11-84e2-295343c8e79a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:51.465588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f852d-9f99-4e11-84e2-295343c8e79a service swift] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:51.470349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:51.471292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:51.471598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:51.471875 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] PATH_INFO: `/v3/users/3cf7a5b535234632bb00a9b87ceb5354` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:51.473070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf7a5b535234632bb00a9b87ceb5354)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:51.475165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:51.678309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf7a5b535234632bb00a9b87ceb5354'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:51.679258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf7a5b535234632bb00a9b87ceb5354'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:51.697464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf7a5b535234632bb00a9b87ceb5354'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:51.698704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf7a5b535234632bb00a9b87ceb5354'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:51.821656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf7a5b535234632bb00a9b87ceb5354'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:51.835179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1152/2317] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:31:51 2022] DELETE /identity/v3/users/3cf7a5b535234632bb00a9b87ceb5354 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:52.075921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6c497-0697-47b1-9f48-f350a4bba4e2 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:52.086908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6c497-0697-47b1-9f48-f350a4bba4e2 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:52.156363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:52.157244 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:52.157556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:52.157835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] PATH_INFO: `/v3/projects/3be6c3cdf062493184ab67ec3ec9940f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:52.159080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be6c3cdf062493184ab67ec3ec9940f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:52.161656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:52.292701 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be6c3cdf062493184ab67ec3ec9940f', 'request_id': 'req-d5833fe7-3c5c-4241-ae12-7b8fc2618891'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.298620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be6c3cdf062493184ab67ec3ec9940f', 'request_id': 'req-d5833fe7-3c5c-4241-ae12-7b8fc2618891'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.384367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be6c3cdf062493184ab67ec3ec9940f', 'request_id': 'req-d5833fe7-3c5c-4241-ae12-7b8fc2618891'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.396497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] The token cache is being invalidate because project 3be6c3cdf062493184ab67ec3ec9940f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:52.399974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.399974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.399974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1166/2318] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:31:52 2022] DELETE /identity/v3/projects/3be6c3cdf062493184ab67ec3ec9940f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:31:52.403552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:52.404977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:52.421010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0409319-dec1-4a52-8990-5a3b0083bbf8 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:52.481229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:52.481999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:52.482284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:52.482594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] PATH_INFO: `/v3/users/1e57847c26ee48108c7285a91a488218` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:52.483840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e57847c26ee48108c7285a91a488218)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:52.485779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:52.716282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e57847c26ee48108c7285a91a488218'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.716928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e57847c26ee48108c7285a91a488218'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.729098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e57847c26ee48108c7285a91a488218'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.729781 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e57847c26ee48108c7285a91a488218'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.821893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e57847c26ee48108c7285a91a488218'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:52.824249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:52.835497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1153/2319] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:31:52 2022] DELETE /identity/v3/users/1e57847c26ee48108c7285a91a488218 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:31:53.171458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:31:53.173034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:31:53.186666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833fe7-3c5c-4241-ae12-7b8fc2618891 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:31:53.224599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:31:53.249965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:31:53.250888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:31:53.251176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:31:53.251453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] PATH_INFO: `/v3/projects/0cf5d50226094a7a8d8e2e968d8a0cb1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:31:53.253742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf5d50226094a7a8d8e2e968d8a0cb1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:31:53.255756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:31:53.400594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf5d50226094a7a8d8e2e968d8a0cb1', 'request_id': 'req-f0473a27-c175-4156-b518-9a4826f423d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:53.405949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf5d50226094a7a8d8e2e968d8a0cb1', 'request_id': 'req-f0473a27-c175-4156-b518-9a4826f423d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:53.518507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf5d50226094a7a8d8e2e968d8a0cb1', 'request_id': 'req-f0473a27-c175-4156-b518-9a4826f423d6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:53.530434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] The token cache is being invalidate because project 0cf5d50226094a7a8d8e2e968d8a0cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:31:53.530740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:53.531391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:31:53.533157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1167/2320] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:31:53 2022] DELETE /identity/v3/projects/0cf5d50226094a7a8d8e2e968d8a0cb1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:03.066891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:03.527256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:13.826746 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:13.828186 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:13.903821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9bb07-d591-4827-a797-9390e5ea5512 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:13.977248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c09146-4b37-4aa0-8b3c-078cca6db45d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:13.977944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c09146-4b37-4aa0-8b3c-078cca6db45d service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:13.978372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c09146-4b37-4aa0-8b3c-078cca6db45d service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:13.978846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c09146-4b37-4aa0-8b3c-078cca6db45d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:13.980683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-17c09146-4b37-4aa0-8b3c-078cca6db45d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:13.993181 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c09146-4b37-4aa0-8b3c-078cca6db45d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:13.994853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c09146-4b37-4aa0-8b3c-078cca6db45d service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:14.005977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1154/2321] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:16.969186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:16.970799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:17.039998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0473a27-c175-4156-b518-9a4826f423d6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:17.044797 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:17.045839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:17.046139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:17.046385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] PATH_INFO: `/v3/users/77104290fa9b4a97850ca238d3df612b` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:17.047633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77104290fa9b4a97850ca238d3df612b)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:17.049580 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:17.259737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77104290fa9b4a97850ca238d3df612b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:17.267527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77104290fa9b4a97850ca238d3df612b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:17.268249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77104290fa9b4a97850ca238d3df612b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:17.268853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77104290fa9b4a97850ca238d3df612b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:17.358881 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77104290fa9b4a97850ca238d3df612b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:17.378459 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1168/2322] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:32:16 2022] DELETE /identity/v3/users/77104290fa9b4a97850ca238d3df612b => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:32:17.383142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553d696-c511-40a0-9029-4044bb564195 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:17.383670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553d696-c511-40a0-9029-4044bb564195 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:17.383775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553d696-c511-40a0-9029-4044bb564195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:17.401234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-4553d696-c511-40a0-9029-4044bb564195 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:17.402577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4553d696-c511-40a0-9029-4044bb564195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:17.465984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1155/2323] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:17.476249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:17.477696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:17.544005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06512e3-3955-408c-a119-f0ba58f908b3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:17.550720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42847d58-062e-4f10-9fce-72dc26ef21c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:17.550720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42847d58-062e-4f10-9fce-72dc26ef21c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:17.550720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42847d58-062e-4f10-9fce-72dc26ef21c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:17.550720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42847d58-062e-4f10-9fce-72dc26ef21c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:17.559930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42847d58-062e-4f10-9fce-72dc26ef21c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:17.561607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42847d58-062e-4f10-9fce-72dc26ef21c4 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:17.573848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1169/2324] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:18.018233 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553d696-c511-40a0-9029-4044bb564195 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:18.025419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-4553d696-c511-40a0-9029-4044bb564195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:18.040560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553d696-c511-40a0-9029-4044bb564195 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:18.048070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:18.048962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:18.049439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:18.049584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] PATH_INFO: `/v3/projects/5d5c453a2fa04819a3cd976ab59eaf0c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:18.050767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d5c453a2fa04819a3cd976ab59eaf0c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:18.052741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:18.120892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:18.137475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d5c453a2fa04819a3cd976ab59eaf0c', 'request_id': 'req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.145469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d5c453a2fa04819a3cd976ab59eaf0c', 'request_id': 'req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.238607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d5c453a2fa04819a3cd976ab59eaf0c', 'request_id': 'req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.251410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] The token cache is being invalidate because project 5d5c453a2fa04819a3cd976ab59eaf0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:32:18.251707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.253180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.255201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1156/2325] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:32:18 2022] DELETE /identity/v3/projects/5d5c453a2fa04819a3cd976ab59eaf0c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:18.258837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42847d58-062e-4f10-9fce-72dc26ef21c4 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:18.269648 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42847d58-062e-4f10-9fce-72dc26ef21c4 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:18.296743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:18.338908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:18.339867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:18.340149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:18.340426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] PATH_INFO: `/v3/users/f8dedf6c411548c895bbf323d07ee6a0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:18.341651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8dedf6c411548c895bbf323d07ee6a0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:18.343815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:18.674778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8dedf6c411548c895bbf323d07ee6a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.682535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8dedf6c411548c895bbf323d07ee6a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.683794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8dedf6c411548c895bbf323d07ee6a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.684423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8dedf6c411548c895bbf323d07ee6a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.872851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-446cc5ae-22f8-495e-8cc9-2f4cdd5db392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8dedf6c411548c895bbf323d07ee6a0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:18.895031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1170/2326] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:32:18 2022] DELETE /identity/v3/users/f8dedf6c411548c895bbf323d07ee6a0 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:19.344454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:19.351621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:19.360581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdac70b-c8a3-4432-9ba5-2ad8cbb9f0ac admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:19.432587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:19.433459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:19.433743 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:19.434049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] PATH_INFO: `/v3/projects/d8972e15a1934b1fb8ce7557f28de5f1` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:19.435298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8972e15a1934b1fb8ce7557f28de5f1)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:19.437591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:19.566668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8972e15a1934b1fb8ce7557f28de5f1', 'request_id': 'req-18a040a0-d4f6-4792-a41f-edbeb50e81c5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:19.572180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8972e15a1934b1fb8ce7557f28de5f1', 'request_id': 'req-18a040a0-d4f6-4792-a41f-edbeb50e81c5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:19.644212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8972e15a1934b1fb8ce7557f28de5f1', 'request_id': 'req-18a040a0-d4f6-4792-a41f-edbeb50e81c5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:19.656439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] The token cache is being invalidate because project d8972e15a1934b1fb8ce7557f28de5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:32:19.656795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:19.657437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-18a040a0-d4f6-4792-a41f-edbeb50e81c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:19.658998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1157/2327] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:32:19 2022] DELETE /identity/v3/projects/d8972e15a1934b1fb8ce7557f28de5f1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:19.664103 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191a505-c9bc-4a2c-aedf-caf8b480f3ec None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:19.664602 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191a505-c9bc-4a2c-aedf-caf8b480f3ec None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:19.664719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191a505-c9bc-4a2c-aedf-caf8b480f3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:19.684135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d191a505-c9bc-4a2c-aedf-caf8b480f3ec None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:19.685400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d191a505-c9bc-4a2c-aedf-caf8b480f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:19.754231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1171/2328] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:32:19.762730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfdb3b-0683-4070-8ba8-46344eef6a2a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:19.763021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfdb3b-0683-4070-8ba8-46344eef6a2a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:19.763302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfdb3b-0683-4070-8ba8-46344eef6a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:19.782315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-46dfdb3b-0683-4070-8ba8-46344eef6a2a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:19.783624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-46dfdb3b-0683-4070-8ba8-46344eef6a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:19.791661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1158/2329] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:19.796071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191a505-c9bc-4a2c-aedf-caf8b480f3ec None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:19.806212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191a505-c9bc-4a2c-aedf-caf8b480f3ec None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:19.810913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002498d-5baf-4521-bb6b-f9b8a4972f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:19.811576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002498d-5baf-4521-bb6b-f9b8a4972f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:19.811867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002498d-5baf-4521-bb6b-f9b8a4972f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:19.812685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002498d-5baf-4521-bb6b-f9b8a4972f98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:19.813415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002498d-5baf-4521-bb6b-f9b8a4972f98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:32:19.813732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002498d-5baf-4521-bb6b-f9b8a4972f98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:19.815430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002498d-5baf-4521-bb6b-f9b8a4972f98 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:19.827477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1172/2330] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:32:19.832544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfdb3b-0683-4070-8ba8-46344eef6a2a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:19.844332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfdb3b-0683-4070-8ba8-46344eef6a2a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:19.849002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b958d09-f05b-4b71-b87f-1b481674bfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:19.849661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958d09-f05b-4b71-b87f-1b481674bfb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:19.849944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958d09-f05b-4b71-b87f-1b481674bfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:19.850242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958d09-f05b-4b71-b87f-1b481674bfb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:19.850996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b958d09-f05b-4b71-b87f-1b481674bfb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:19.852889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b958d09-f05b-4b71-b87f-1b481674bfb8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:19.938745 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1159/2331] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:32:19 2022] POST /identity/v3/projects => generated 361 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:19.943030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002498d-5baf-4521-bb6b-f9b8a4972f98 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:19.955130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002498d-5baf-4521-bb6b-f9b8a4972f98 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:19.983115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:20.017735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e332b-5e56-46a4-90b2-01310b4a14da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:20.018461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e332b-5e56-46a4-90b2-01310b4a14da admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:20.018758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e332b-5e56-46a4-90b2-01310b4a14da admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:20.019037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e332b-5e56-46a4-90b2-01310b4a14da admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:20.019735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e332b-5e56-46a4-90b2-01310b4a14da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:20.021812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e332b-5e56-46a4-90b2-01310b4a14da admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:20.107209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1173/2332] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:32:19 2022] POST /identity/v3/users => generated 281 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:20.112177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b958d09-f05b-4b71-b87f-1b481674bfb8 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:20.122604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b958d09-f05b-4b71-b87f-1b481674bfb8 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:20.127950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb8903-afe4-4bd6-a419-23ee5a4fe679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:20.128578 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb8903-afe4-4bd6-a419-23ee5a4fe679 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:20.128862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb8903-afe4-4bd6-a419-23ee5a4fe679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:20.129138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb8903-afe4-4bd6-a419-23ee5a4fe679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:20.129893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb8903-afe4-4bd6-a419-23ee5a4fe679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:20.131522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb8903-afe4-4bd6-a419-23ee5a4fe679 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:20.141947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1160/2333] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:20.146027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e332b-5e56-46a4-90b2-01310b4a14da admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:20.157106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e332b-5e56-46a4-90b2-01310b4a14da admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:20.161758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cebc00-e9c1-4f43-8194-1931a2a8400e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:20.162507 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cebc00-e9c1-4f43-8194-1931a2a8400e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:20.162793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cebc00-e9c1-4f43-8194-1931a2a8400e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:20.163068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cebc00-e9c1-4f43-8194-1931a2a8400e admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:20.163791 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cebc00-e9c1-4f43-8194-1931a2a8400e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:20.165398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cebc00-e9c1-4f43-8194-1931a2a8400e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:20.177288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1174/2334] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:20.181823 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb8903-afe4-4bd6-a419-23ee5a4fe679 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:20.192695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb8903-afe4-4bd6-a419-23ee5a4fe679 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:20.197317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbad710-b85b-43a9-9366-883c4a71f330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:20.198190 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad710-b85b-43a9-9366-883c4a71f330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:20.198501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad710-b85b-43a9-9366-883c4a71f330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:20.198778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbad710-b85b-43a9-9366-883c4a71f330 admin admin] PATH_INFO: `/v3/projects/14268abf77424765942af96a327915c7/users/6c1f174dd96e4577bdfd47347a96c057/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:20.201032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:20.214239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbad710-b85b-43a9-9366-883c4a71f330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14268abf77424765942af96a327915c7, user_id=6c1f174dd96e4577bdfd47347a96c057, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:20.216941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbad710-b85b-43a9-9366-883c4a71f330 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:20.284122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1161/2335] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:32:20 2022] PUT /identity/v3/projects/14268abf77424765942af96a327915c7/users/6c1f174dd96e4577bdfd47347a96c057/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:20.295629 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb8c65-7950-4d1a-9e9e-eac21256c985 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:20.295926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb8c65-7950-4d1a-9e9e-eac21256c985 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:20.296212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb8c65-7950-4d1a-9e9e-eac21256c985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:20.324065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b6bb8c65-7950-4d1a-9e9e-eac21256c985 None None] MFA Rules not processed for user `6c1f174dd96e4577bdfd47347a96c057`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:20.325272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b6bb8c65-7950-4d1a-9e9e-eac21256c985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:20.393039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1175/2336] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:20.436612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbad710-b85b-43a9-9366-883c4a71f330 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:20.439721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7dbad710-b85b-43a9-9366-883c4a71f330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:20.509221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbad710-b85b-43a9-9366-883c4a71f330 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:20.512892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087537d5-17fd-46f6-9462-106526804be9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:20.513529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087537d5-17fd-46f6-9462-106526804be9 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:20.513810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087537d5-17fd-46f6-9462-106526804be9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:20.514084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087537d5-17fd-46f6-9462-106526804be9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:20.523078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087537d5-17fd-46f6-9462-106526804be9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:20.528005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087537d5-17fd-46f6-9462-106526804be9 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:20.534008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1162/2337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:20.551384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb8c65-7950-4d1a-9e9e-eac21256c985 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:20.613503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb8c65-7950-4d1a-9e9e-eac21256c985 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:20.618072 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:20.618986 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:20.619278 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:20.619556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] PATH_INFO: `/v3/users/6c1f174dd96e4577bdfd47347a96c057` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:20.620788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1f174dd96e4577bdfd47347a96c057)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:20.622752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:20.829835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:20.835770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1f174dd96e4577bdfd47347a96c057'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:20.843877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1f174dd96e4577bdfd47347a96c057'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:20.844089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1f174dd96e4577bdfd47347a96c057'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:20.844601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1f174dd96e4577bdfd47347a96c057'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:20.929495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1f174dd96e4577bdfd47347a96c057'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:20.948031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1176/2338] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:32:20 2022] DELETE /identity/v3/users/6c1f174dd96e4577bdfd47347a96c057 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:20.954280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ad74b-d127-4606-b6d5-293bacdfbd44 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:20.954280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ad74b-d127-4606-b6d5-293bacdfbd44 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:20.954280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ad74b-d127-4606-b6d5-293bacdfbd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:20.971056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-201ad74b-d127-4606-b6d5-293bacdfbd44 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:20.972278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-201ad74b-d127-4606-b6d5-293bacdfbd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:21.036622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1163/2339] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:21.047412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:21.049691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:21.112206 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015999c-f036-4279-9e69-69aff947e1d0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:21.115883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225b4f6-c9b7-4330-a1d7-0209ffb3a3a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:21.116530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225b4f6-c9b7-4330-a1d7-0209ffb3a3a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:21.116817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225b4f6-c9b7-4330-a1d7-0209ffb3a3a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:21.117096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225b4f6-c9b7-4330-a1d7-0209ffb3a3a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:21.126200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225b4f6-c9b7-4330-a1d7-0209ffb3a3a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:21.127673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225b4f6-c9b7-4330-a1d7-0209ffb3a3a0 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:21.137078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1177/2340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:21.601008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ad74b-d127-4606-b6d5-293bacdfbd44 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:21.602576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-201ad74b-d127-4606-b6d5-293bacdfbd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:21.616194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ad74b-d127-4606-b6d5-293bacdfbd44 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:21.626599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:21.626599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:21.626599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:21.626599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] PATH_INFO: `/v3/projects/14268abf77424765942af96a327915c7` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:21.627505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14268abf77424765942af96a327915c7)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:21.627505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:21.753769 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14268abf77424765942af96a327915c7', 'request_id': 'req-42512f4f-a60b-4499-a6c6-b9768e6aac39'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.759397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14268abf77424765942af96a327915c7', 'request_id': 'req-42512f4f-a60b-4499-a6c6-b9768e6aac39'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.844572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14268abf77424765942af96a327915c7', 'request_id': 'req-42512f4f-a60b-4499-a6c6-b9768e6aac39'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.856063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] The token cache is being invalidate because project 14268abf77424765942af96a327915c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:32:21.856353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.856977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-42512f4f-a60b-4499-a6c6-b9768e6aac39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:21.858568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1164/2341] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:32:21 2022] DELETE /identity/v3/projects/14268abf77424765942af96a327915c7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:21.863463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeb5a4-a2fc-4559-86bc-52d0e448aebd None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:21.863940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeb5a4-a2fc-4559-86bc-52d0e448aebd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:21.864036 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddeb5a4-a2fc-4559-86bc-52d0e448aebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:21.881854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6ddeb5a4-a2fc-4559-86bc-52d0e448aebd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:21.883081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6ddeb5a4-a2fc-4559-86bc-52d0e448aebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:21.942828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1178/2342] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:21.950648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d84ad9-d3ad-4261-aacf-76decf30ed77 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:21.950944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d84ad9-d3ad-4261-aacf-76decf30ed77 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:21.951224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d84ad9-d3ad-4261-aacf-76decf30ed77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:21.970761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-a3d84ad9-d3ad-4261-aacf-76decf30ed77 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:21.972003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a3d84ad9-d3ad-4261-aacf-76decf30ed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:21.980153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1165/2343] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:21.990634 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddeb5a4-a2fc-4559-86bc-52d0e448aebd None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:22.000937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddeb5a4-a2fc-4559-86bc-52d0e448aebd None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:22.005554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d328053-47b1-4311-b417-15e8cef0d808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:22.006198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d328053-47b1-4311-b417-15e8cef0d808 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.006519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d328053-47b1-4311-b417-15e8cef0d808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.006798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d328053-47b1-4311-b417-15e8cef0d808 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.007549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d328053-47b1-4311-b417-15e8cef0d808 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:32:22.007918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d328053-47b1-4311-b417-15e8cef0d808 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:22.009504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d328053-47b1-4311-b417-15e8cef0d808 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:22.022355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1179/2344] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:32:22.026663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d84ad9-d3ad-4261-aacf-76decf30ed77 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:22.032906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:22.036909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d84ad9-d3ad-4261-aacf-76decf30ed77 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:22.041990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5814e17-8241-43db-a9d1-756016981c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:22.042686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5814e17-8241-43db-a9d1-756016981c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.042992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5814e17-8241-43db-a9d1-756016981c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.043265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5814e17-8241-43db-a9d1-756016981c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.043971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5814e17-8241-43db-a9d1-756016981c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:22.045974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5814e17-8241-43db-a9d1-756016981c02 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:22.119324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1166/2345] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:32:22 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:22.123552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d328053-47b1-4311-b417-15e8cef0d808 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:22.134361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d328053-47b1-4311-b417-15e8cef0d808 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:22.182783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:22.196237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e05fea-f601-45ee-b446-8fb6aaf7eca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:22.197041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e05fea-f601-45ee-b446-8fb6aaf7eca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.197324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e05fea-f601-45ee-b446-8fb6aaf7eca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.197606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e05fea-f601-45ee-b446-8fb6aaf7eca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.198303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e05fea-f601-45ee-b446-8fb6aaf7eca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:22.200252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e05fea-f601-45ee-b446-8fb6aaf7eca7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:22.313070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1180/2346] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:32:22 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:22.317573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5814e17-8241-43db-a9d1-756016981c02 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:22.328656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5814e17-8241-43db-a9d1-756016981c02 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:22.333690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08eaa4-f47b-453d-832f-41d6dd25fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:22.334353 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08eaa4-f47b-453d-832f-41d6dd25fb12 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.334673 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08eaa4-f47b-453d-832f-41d6dd25fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.334947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08eaa4-f47b-453d-832f-41d6dd25fb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.335672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08eaa4-f47b-453d-832f-41d6dd25fb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:22.337290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08eaa4-f47b-453d-832f-41d6dd25fb12 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:22.348114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1167/2347] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:22.352314 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e05fea-f601-45ee-b446-8fb6aaf7eca7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:22.363151 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e05fea-f601-45ee-b446-8fb6aaf7eca7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:22.367838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54eefd-6b75-4322-92a7-93204ddc05aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:22.368731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54eefd-6b75-4322-92a7-93204ddc05aa admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.369196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54eefd-6b75-4322-92a7-93204ddc05aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.369846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54eefd-6b75-4322-92a7-93204ddc05aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.370604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54eefd-6b75-4322-92a7-93204ddc05aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:22.372242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54eefd-6b75-4322-92a7-93204ddc05aa admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:22.383296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1181/2348] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:22.387580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08eaa4-f47b-453d-832f-41d6dd25fb12 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:22.398539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08eaa4-f47b-453d-832f-41d6dd25fb12 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:22.403251 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc40a14-b917-433a-a4d7-0f38d0e61edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:22.404137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc40a14-b917-433a-a4d7-0f38d0e61edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.404423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc40a14-b917-433a-a4d7-0f38d0e61edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.404703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc40a14-b917-433a-a4d7-0f38d0e61edf admin admin] PATH_INFO: `/v3/projects/a1bf7476eb6b4443adc5828e30f99f8d/users/d558bf98bf5b4e36895289a28a5d31c6/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.421102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc40a14-b917-433a-a4d7-0f38d0e61edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1bf7476eb6b4443adc5828e30f99f8d, user_id=d558bf98bf5b4e36895289a28a5d31c6, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:22.423892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc40a14-b917-433a-a4d7-0f38d0e61edf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:22.508134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1168/2349] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:32:22 2022] PUT /identity/v3/projects/a1bf7476eb6b4443adc5828e30f99f8d/users/d558bf98bf5b4e36895289a28a5d31c6/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:22.520591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b9feb-2520-4007-9b5c-8fda82380f9c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.520889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b9feb-2520-4007-9b5c-8fda82380f9c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.521178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586b9feb-2520-4007-9b5c-8fda82380f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.550291 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-586b9feb-2520-4007-9b5c-8fda82380f9c None None] MFA Rules not processed for user `d558bf98bf5b4e36895289a28a5d31c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:22.551563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-586b9feb-2520-4007-9b5c-8fda82380f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:22.616535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1182/2350] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:22.628375 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc40a14-b917-433a-a4d7-0f38d0e61edf admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:22.630185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9bc40a14-b917-433a-a4d7-0f38d0e61edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:22.687872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc40a14-b917-433a-a4d7-0f38d0e61edf admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:22.691431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17332233-c040-47f4-ad84-11c2f9413a13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:22.692079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17332233-c040-47f4-ad84-11c2f9413a13 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:22.692365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17332233-c040-47f4-ad84-11c2f9413a13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:22.692646 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17332233-c040-47f4-ad84-11c2f9413a13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:22.701028 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17332233-c040-47f4-ad84-11c2f9413a13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:22.703006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17332233-c040-47f4-ad84-11c2f9413a13 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:22.711843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1169/2351] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:25.795955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b9feb-2520-4007-9b5c-8fda82380f9c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:25.863511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586b9feb-2520-4007-9b5c-8fda82380f9c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:25.867935 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:25.868803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:25.869137 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:25.869417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] PATH_INFO: `/v3/users/d558bf98bf5b4e36895289a28a5d31c6` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:25.870661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d558bf98bf5b4e36895289a28a5d31c6)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:25.872599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:26.052347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd558bf98bf5b4e36895289a28a5d31c6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:26.059627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd558bf98bf5b4e36895289a28a5d31c6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:26.060300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd558bf98bf5b4e36895289a28a5d31c6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:26.062519 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd558bf98bf5b4e36895289a28a5d31c6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:26.062641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:26.129287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd558bf98bf5b4e36895289a28a5d31c6'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:26.147646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1183/2352] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:32:25 2022] DELETE /identity/v3/users/d558bf98bf5b4e36895289a28a5d31c6 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:26.152563 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e930676-73e9-439e-864b-7beb9ed312ba None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:26.153037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e930676-73e9-439e-864b-7beb9ed312ba None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:26.153154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e930676-73e9-439e-864b-7beb9ed312ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:26.170018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5e930676-73e9-439e-864b-7beb9ed312ba None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:26.171326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5e930676-73e9-439e-864b-7beb9ed312ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:26.231073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1170/2353] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:26.242564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:26.243959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:26.305154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4809e-727d-4fde-871d-91b462f4461a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:26.308761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbf8d2-fdbe-4af4-ab3d-9e758a72872a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:26.309404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbf8d2-fdbe-4af4-ab3d-9e758a72872a service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:26.309690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbf8d2-fdbe-4af4-ab3d-9e758a72872a service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:26.309964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbf8d2-fdbe-4af4-ab3d-9e758a72872a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:26.319738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbf8d2-fdbe-4af4-ab3d-9e758a72872a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:26.321165 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbf8d2-fdbe-4af4-ab3d-9e758a72872a service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:26.331183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1184/2354] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:26.823114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e930676-73e9-439e-864b-7beb9ed312ba None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:26.824547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5e930676-73e9-439e-864b-7beb9ed312ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:26.839566 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e930676-73e9-439e-864b-7beb9ed312ba None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:26.843078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:26.843950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:26.844229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:26.844520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] PATH_INFO: `/v3/projects/a1bf7476eb6b4443adc5828e30f99f8d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:26.846494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1bf7476eb6b4443adc5828e30f99f8d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:26.848438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:26.989941 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:26.993558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1bf7476eb6b4443adc5828e30f99f8d', 'request_id': 'req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:26.998965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1bf7476eb6b4443adc5828e30f99f8d', 'request_id': 'req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:27.146446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1bf7476eb6b4443adc5828e30f99f8d', 'request_id': 'req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:27.158821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] The token cache is being invalidate because project a1bf7476eb6b4443adc5828e30f99f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:32:27.166983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:27.166983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-7c2244b8-2c9d-4474-a58d-26ea04ac50f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:27.166983 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1171/2355] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:32:26 2022] DELETE /identity/v3/projects/a1bf7476eb6b4443adc5828e30f99f8d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:27.167512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3087ec-c097-4001-893d-8a10bb05fbc4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:27.167512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3087ec-c097-4001-893d-8a10bb05fbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:27.167512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3087ec-c097-4001-893d-8a10bb05fbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:27.186707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7e3087ec-c097-4001-893d-8a10bb05fbc4 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:27.188092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7e3087ec-c097-4001-893d-8a10bb05fbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:27.249282 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1185/2356] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:27.259003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ece5ec-fa6b-448e-8a56-0bf0c30fc937 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:27.259270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ece5ec-fa6b-448e-8a56-0bf0c30fc937 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:27.259556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ece5ec-fa6b-448e-8a56-0bf0c30fc937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:27.279150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-26ece5ec-fa6b-448e-8a56-0bf0c30fc937 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:27.280360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-26ece5ec-fa6b-448e-8a56-0bf0c30fc937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:27.288688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1172/2357] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:27.292943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3087ec-c097-4001-893d-8a10bb05fbc4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:27.303245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3087ec-c097-4001-893d-8a10bb05fbc4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:27.308077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea961d3-f952-42c9-8e4f-8b49e0454b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:27.308736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea961d3-f952-42c9-8e4f-8b49e0454b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:27.309019 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea961d3-f952-42c9-8e4f-8b49e0454b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:27.309294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea961d3-f952-42c9-8e4f-8b49e0454b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:27.310038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea961d3-f952-42c9-8e4f-8b49e0454b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:32:27.310348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea961d3-f952-42c9-8e4f-8b49e0454b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:27.312011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea961d3-f952-42c9-8e4f-8b49e0454b24 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:27.325064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1186/2358] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:32:27.330277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ece5ec-fa6b-448e-8a56-0bf0c30fc937 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:27.339882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ece5ec-fa6b-448e-8a56-0bf0c30fc937 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:27.344436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6b6b7-a691-459c-ba2b-5ed6a5403b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:27.346076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6b6b7-a691-459c-ba2b-5ed6a5403b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:27.346360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6b6b7-a691-459c-ba2b-5ed6a5403b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:27.346668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6b6b7-a691-459c-ba2b-5ed6a5403b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:27.347370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6b6b7-a691-459c-ba2b-5ed6a5403b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:27.349268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6b6b7-a691-459c-ba2b-5ed6a5403b2b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:27.428779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1173/2359] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:32:27 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:27.433255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea961d3-f952-42c9-8e4f-8b49e0454b24 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:27.449252 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea961d3-f952-42c9-8e4f-8b49e0454b24 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:27.508503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4389e-9996-4684-b112-af996d094c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:27.509171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4389e-9996-4684-b112-af996d094c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:27.509449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4389e-9996-4684-b112-af996d094c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:27.509725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4389e-9996-4684-b112-af996d094c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:27.510439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4389e-9996-4684-b112-af996d094c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:27.512477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4389e-9996-4684-b112-af996d094c19 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:27.521978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:27.634496 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1187/2360] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:32:27 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:27.638657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6b6b7-a691-459c-ba2b-5ed6a5403b2b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:27.649658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6b6b7-a691-459c-ba2b-5ed6a5403b2b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:27.654226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee75b5-05b1-45bc-b4ae-13b9470df1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:27.654917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee75b5-05b1-45bc-b4ae-13b9470df1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:27.655195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee75b5-05b1-45bc-b4ae-13b9470df1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:27.655475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee75b5-05b1-45bc-b4ae-13b9470df1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:27.656195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee75b5-05b1-45bc-b4ae-13b9470df1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:27.658676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee75b5-05b1-45bc-b4ae-13b9470df1bd admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:27.669242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1174/2361] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:32:27.675710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4389e-9996-4684-b112-af996d094c19 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:27.686640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4389e-9996-4684-b112-af996d094c19 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:27.691614 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da84f1b-d236-481c-90c2-351cf3c1c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:27.692281 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da84f1b-d236-481c-90c2-351cf3c1c470 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:27.692570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da84f1b-d236-481c-90c2-351cf3c1c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:27.692858 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da84f1b-d236-481c-90c2-351cf3c1c470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:27.693592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da84f1b-d236-481c-90c2-351cf3c1c470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:27.695238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da84f1b-d236-481c-90c2-351cf3c1c470 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:27.707255 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1188/2362] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:27.711403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee75b5-05b1-45bc-b4ae-13b9470df1bd admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:27.722462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee75b5-05b1-45bc-b4ae-13b9470df1bd admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:27.727306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43d345-8d1c-46ea-873f-889ea2d394d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:27.729962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43d345-8d1c-46ea-873f-889ea2d394d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:27.729962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43d345-8d1c-46ea-873f-889ea2d394d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:27.729962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43d345-8d1c-46ea-873f-889ea2d394d4 admin admin] PATH_INFO: `/v3/projects/d5a46bc8a612457daadac57bea36c05a/users/d769d9d0d8c448cea10077a32c051cd1/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:27.744140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43d345-8d1c-46ea-873f-889ea2d394d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a46bc8a612457daadac57bea36c05a, user_id=d769d9d0d8c448cea10077a32c051cd1, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:27.747385 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43d345-8d1c-46ea-873f-889ea2d394d4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:27.855357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1175/2363] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:32:27 2022] PUT /identity/v3/projects/d5a46bc8a612457daadac57bea36c05a/users/d769d9d0d8c448cea10077a32c051cd1/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:32:27.867457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60151b1e-2bb3-4afb-bee5-d10d0ec7610a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:27.867784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60151b1e-2bb3-4afb-bee5-d10d0ec7610a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:27.868117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60151b1e-2bb3-4afb-bee5-d10d0ec7610a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:27.894999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-60151b1e-2bb3-4afb-bee5-d10d0ec7610a None None] MFA Rules not processed for user `d769d9d0d8c448cea10077a32c051cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:27.896217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-60151b1e-2bb3-4afb-bee5-d10d0ec7610a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:27.962267 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1189/2364] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:27.977685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43d345-8d1c-46ea-873f-889ea2d394d4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:27.979536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fa43d345-8d1c-46ea-873f-889ea2d394d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:28.040478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43d345-8d1c-46ea-873f-889ea2d394d4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:28.044073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c79f0b-3a2f-45e5-b47b-837291504ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:28.044830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c79f0b-3a2f-45e5-b47b-837291504ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:28.045115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c79f0b-3a2f-45e5-b47b-837291504ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:28.045391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c79f0b-3a2f-45e5-b47b-837291504ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:28.054012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c79f0b-3a2f-45e5-b47b-837291504ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:28.055811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c79f0b-3a2f-45e5-b47b-837291504ad2 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:28.065703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1176/2365] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:28.091004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60151b1e-2bb3-4afb-bee5-d10d0ec7610a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:28.160609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60151b1e-2bb3-4afb-bee5-d10d0ec7610a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:28.165306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:28.166174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:28.166481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:28.166761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] PATH_INFO: `/v3/users/d769d9d0d8c448cea10077a32c051cd1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:28.167940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d769d9d0d8c448cea10077a32c051cd1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:28.169851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:28.348238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd769d9d0d8c448cea10077a32c051cd1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:28.355277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd769d9d0d8c448cea10077a32c051cd1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:28.356062 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd769d9d0d8c448cea10077a32c051cd1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:28.356676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd769d9d0d8c448cea10077a32c051cd1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:28.432010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd769d9d0d8c448cea10077a32c051cd1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:28.441403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:28.451328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1190/2366] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:32:28 2022] DELETE /identity/v3/users/d769d9d0d8c448cea10077a32c051cd1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:28.456054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351517a-5405-408d-be7b-398d69b23cb7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:28.456527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351517a-5405-408d-be7b-398d69b23cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:28.456642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351517a-5405-408d-be7b-398d69b23cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:28.473596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e351517a-5405-408d-be7b-398d69b23cb7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:28.478566 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e351517a-5405-408d-be7b-398d69b23cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:28.541247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1177/2367] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:28.553230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:28.555656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:28.619093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7790c77-09aa-4fa7-b503-0a78c947b4b2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:28.623078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d77c19-1370-4bcc-9732-e7e106d9c5c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:28.623757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d77c19-1370-4bcc-9732-e7e106d9c5c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:28.624047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d77c19-1370-4bcc-9732-e7e106d9c5c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:28.624325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d77c19-1370-4bcc-9732-e7e106d9c5c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:28.636056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d77c19-1370-4bcc-9732-e7e106d9c5c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:28.637524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d77c19-1370-4bcc-9732-e7e106d9c5c7 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:28.649168 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1191/2368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:29.091376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351517a-5405-408d-be7b-398d69b23cb7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:29.092777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e351517a-5405-408d-be7b-398d69b23cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:29.105282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351517a-5405-408d-be7b-398d69b23cb7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:29.110048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:29.111130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.111429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.111712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] PATH_INFO: `/v3/projects/d5a46bc8a612457daadac57bea36c05a` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:29.112911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a46bc8a612457daadac57bea36c05a)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:29.114869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:29.121568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:29.228924 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a46bc8a612457daadac57bea36c05a', 'request_id': 'req-1f96a974-650e-4e3c-822b-57fcbb7230ef'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:29.235066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a46bc8a612457daadac57bea36c05a', 'request_id': 'req-1f96a974-650e-4e3c-822b-57fcbb7230ef'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:29.322820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a46bc8a612457daadac57bea36c05a', 'request_id': 'req-1f96a974-650e-4e3c-822b-57fcbb7230ef'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:29.335790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] The token cache is being invalidate because project d5a46bc8a612457daadac57bea36c05a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:32:29.336076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:29.336711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-1f96a974-650e-4e3c-822b-57fcbb7230ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:32:29.338231 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1178/2369] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:32:29 2022] DELETE /identity/v3/projects/d5a46bc8a612457daadac57bea36c05a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:32:29.343840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fac9b4-9175-4f87-a78b-9fc04a693b80 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.349126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fac9b4-9175-4f87-a78b-9fc04a693b80 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.349126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fac9b4-9175-4f87-a78b-9fc04a693b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:29.363317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-42fac9b4-9175-4f87-a78b-9fc04a693b80 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:29.364499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-42fac9b4-9175-4f87-a78b-9fc04a693b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:29.424970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1192/2370] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:29.433340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a5a3e-3bdb-4668-88ff-be25112e268e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.433634 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a5a3e-3bdb-4668-88ff-be25112e268e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.433921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a5a3e-3bdb-4668-88ff-be25112e268e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:29.453083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d50a5a3e-3bdb-4668-88ff-be25112e268e None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:29.454452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d50a5a3e-3bdb-4668-88ff-be25112e268e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:29.462368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1179/2371] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:29.466683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fac9b4-9175-4f87-a78b-9fc04a693b80 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:29.477357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fac9b4-9175-4f87-a78b-9fc04a693b80 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:29.482055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be80702-58f6-4554-9c68-b8016c654f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:29.482780 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be80702-58f6-4554-9c68-b8016c654f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.483066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be80702-58f6-4554-9c68-b8016c654f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.483347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be80702-58f6-4554-9c68-b8016c654f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:29.484071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be80702-58f6-4554-9c68-b8016c654f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:32:29.484384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be80702-58f6-4554-9c68-b8016c654f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:29.486652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be80702-58f6-4554-9c68-b8016c654f1d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:29.498584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1193/2372] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:32:29.502960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a5a3e-3bdb-4668-88ff-be25112e268e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:29.512192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a5a3e-3bdb-4668-88ff-be25112e268e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:29.517382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4594-29f4-4f67-be62-dea395779b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:29.518242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4594-29f4-4f67-be62-dea395779b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.518566 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4594-29f4-4f67-be62-dea395779b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.518844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4594-29f4-4f67-be62-dea395779b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:29.519560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f4594-29f4-4f67-be62-dea395779b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:29.521497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f4594-29f4-4f67-be62-dea395779b1f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:29.612214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1180/2373] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:32:29 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:32:29.616650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be80702-58f6-4554-9c68-b8016c654f1d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:29.627582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be80702-58f6-4554-9c68-b8016c654f1d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:29.684805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6331c5f-1b1d-4952-861f-2fbbd6d434fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:29.685390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6331c5f-1b1d-4952-861f-2fbbd6d434fa admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.685671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6331c5f-1b1d-4952-861f-2fbbd6d434fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.685946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6331c5f-1b1d-4952-861f-2fbbd6d434fa admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:29.686924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6331c5f-1b1d-4952-861f-2fbbd6d434fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:29.689315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6331c5f-1b1d-4952-861f-2fbbd6d434fa admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:29.805978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1194/2374] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:32:29 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:32:29.810668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4594-29f4-4f67-be62-dea395779b1f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:29.822033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4594-29f4-4f67-be62-dea395779b1f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:29.827309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422accb-06a9-4622-94b9-3a062d5f1f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:29.828122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422accb-06a9-4622-94b9-3a062d5f1f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.828410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422accb-06a9-4622-94b9-3a062d5f1f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.828692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422accb-06a9-4622-94b9-3a062d5f1f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:29.830273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422accb-06a9-4622-94b9-3a062d5f1f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:29.832095 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422accb-06a9-4622-94b9-3a062d5f1f46 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:29.843405 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1181/2375] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:29.849138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6331c5f-1b1d-4952-861f-2fbbd6d434fa admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:29.867140 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6331c5f-1b1d-4952-861f-2fbbd6d434fa admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:29.867992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16946ec-cc28-43f7-8ff4-b343e350c9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:29.868994 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16946ec-cc28-43f7-8ff4-b343e350c9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.869498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16946ec-cc28-43f7-8ff4-b343e350c9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.869964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16946ec-cc28-43f7-8ff4-b343e350c9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:29.871225 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16946ec-cc28-43f7-8ff4-b343e350c9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:29.872961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16946ec-cc28-43f7-8ff4-b343e350c9e1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:29.881816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:29.890321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1195/2376] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:32:29.897775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422accb-06a9-4622-94b9-3a062d5f1f46 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:29.909468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422accb-06a9-4622-94b9-3a062d5f1f46 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:29.914637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf3592-1c7a-40bb-a02d-f83f6f18f6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:29.915529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf3592-1c7a-40bb-a02d-f83f6f18f6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.915818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf3592-1c7a-40bb-a02d-f83f6f18f6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.916098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf3592-1c7a-40bb-a02d-f83f6f18f6bf admin admin] PATH_INFO: `/v3/projects/886ee445e38e4c31bd4d4d014a7c4c7c/users/8f609cfe0d7a4ded8a622c65d8690f7f/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:29.932627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf3592-1c7a-40bb-a02d-f83f6f18f6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=886ee445e38e4c31bd4d4d014a7c4c7c, user_id=8f609cfe0d7a4ded8a622c65d8690f7f, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:29.935434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf3592-1c7a-40bb-a02d-f83f6f18f6bf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:29.987808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1182/2377] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:32:29 2022] PUT /identity/v3/projects/886ee445e38e4c31bd4d4d014a7c4c7c/users/8f609cfe0d7a4ded8a622c65d8690f7f/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:32:29.992972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c2660-3681-43fe-945b-482467688dd5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:29.993265 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c2660-3681-43fe-945b-482467688dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:29.993549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c2660-3681-43fe-945b-482467688dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:30.012545 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-046c2660-3681-43fe-945b-482467688dd5 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:30.013710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-046c2660-3681-43fe-945b-482467688dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:30.077807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1196/2378] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:30.088092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf3592-1c7a-40bb-a02d-f83f6f18f6bf admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:30.089809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f6cf3592-1c7a-40bb-a02d-f83f6f18f6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:30.151444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf3592-1c7a-40bb-a02d-f83f6f18f6bf admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:30.155205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26dd5-4ce9-44e6-984f-f01b902e6b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:30.155853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26dd5-4ce9-44e6-984f-f01b902e6b94 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:30.156141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26dd5-4ce9-44e6-984f-f01b902e6b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:30.156422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26dd5-4ce9-44e6-984f-f01b902e6b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:30.167750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc26dd5-4ce9-44e6-984f-f01b902e6b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:30.169191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc26dd5-4ce9-44e6-984f-f01b902e6b94 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:30.179148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1183/2379] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:31.754325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f996f-0e5e-446b-a3ff-a59dae0cdec4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:31.754981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f996f-0e5e-446b-a3ff-a59dae0cdec4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:31.754981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f996f-0e5e-446b-a3ff-a59dae0cdec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:31.783735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-760f996f-0e5e-446b-a3ff-a59dae0cdec4 None None] MFA Rules not processed for user `8f609cfe0d7a4ded8a622c65d8690f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:31.784970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-760f996f-0e5e-446b-a3ff-a59dae0cdec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:31.856562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1197/2380] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:31.867644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26dd5-4ce9-44e6-984f-f01b902e6b94 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:32:31.869033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2bc26dd5-4ce9-44e6-984f-f01b902e6b94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:31.905668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:31.926942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26dd5-4ce9-44e6-984f-f01b902e6b94 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:32:31.930847 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db853f3-b24a-4d90-b5f1-1c6bdc28dccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:32:31.931485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db853f3-b24a-4d90-b5f1-1c6bdc28dccb service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:31.931770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db853f3-b24a-4d90-b5f1-1c6bdc28dccb service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:31.932047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db853f3-b24a-4d90-b5f1-1c6bdc28dccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:31.942022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db853f3-b24a-4d90-b5f1-1c6bdc28dccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:32:31.943514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db853f3-b24a-4d90-b5f1-1c6bdc28dccb service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:32:31.952630 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1184/2381] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:32:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:32:34.863421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370885a5-eaae-4b5e-aae4-8b9947a448e3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:32:34.863991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370885a5-eaae-4b5e-aae4-8b9947a448e3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:32:34.863991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370885a5-eaae-4b5e-aae4-8b9947a448e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:32:34.882098 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-370885a5-eaae-4b5e-aae4-8b9947a448e3 None None] MFA Rules not processed for user `8f609cfe0d7a4ded8a622c65d8690f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:32:34.883362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-370885a5-eaae-4b5e-aae4-8b9947a448e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:32:34.890339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1198/2382] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:32:41.986109 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=11 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:32:44.876777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=62 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:32.632281 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8587da9-6e5b-49d4-a1c6-6d24c84332b9 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:33:32.632849 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8587da9-6e5b-49d4-a1c6-6d24c84332b9 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:33:32.632956 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8587da9-6e5b-49d4-a1c6-6d24c84332b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:33:32.651271 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e8587da9-6e5b-49d4-a1c6-6d24c84332b9 None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:33:32.652530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e8587da9-6e5b-49d4-a1c6-6d24c84332b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:32.715833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1185/2383] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:33:32.886668 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370885a5-eaae-4b5e-aae4-8b9947a448e3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:33:32.888229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-370885a5-eaae-4b5e-aae4-8b9947a448e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:32.965525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370885a5-eaae-4b5e-aae4-8b9947a448e3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:33:32.970761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715f7f0-4b8d-418f-9501-00affef8bdf8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:33:32.971418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715f7f0-4b8d-418f-9501-00affef8bdf8 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:33:32.971710 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715f7f0-4b8d-418f-9501-00affef8bdf8 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:33:32.972004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715f7f0-4b8d-418f-9501-00affef8bdf8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:33:32.980983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715f7f0-4b8d-418f-9501-00affef8bdf8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:33:32.982642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715f7f0-4b8d-418f-9501-00affef8bdf8 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:33:32.990859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1199/2384] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:33:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 12:33:42.708504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=25 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:42.987688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=32 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:33:52.043868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa622c-1ef2-4c2c-8e13-62098714db5f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:33:52.044411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa622c-1ef2-4c2c-8e13-62098714db5f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:33:52.044411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa622c-1ef2-4c2c-8e13-62098714db5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:33:52.067306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-79fa622c-1ef2-4c2c-8e13-62098714db5f None None] MFA Rules not processed for user `f0a98f1c068843a684f6765d41cc4e49`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:33:52.067306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-79fa622c-1ef2-4c2c-8e13-62098714db5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:52.120227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1186/2385] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:33:57.773899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715f7f0-4b8d-418f-9501-00affef8bdf8 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:33:57.775371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2715f7f0-4b8d-418f-9501-00affef8bdf8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:57.833312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715f7f0-4b8d-418f-9501-00affef8bdf8 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:33:57.837035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977ba5e-dad7-4749-8075-fb081c7caf4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9969632e796345fa9671a45cf61ede3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:33:57.837676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977ba5e-dad7-4749-8075-fb081c7caf4d service glance] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:33:57.837957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977ba5e-dad7-4749-8075-fb081c7caf4d service glance] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:33:57.838232 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977ba5e-dad7-4749-8075-fb081c7caf4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:33:57.840918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a977ba5e-dad7-4749-8075-fb081c7caf4d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:57.915561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977ba5e-dad7-4749-8075-fb081c7caf4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:33:57.917009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977ba5e-dad7-4749-8075-fb081c7caf4d service glance] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:33:57.928150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1200/2386] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:33:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:33:58.142074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa622c-1ef2-4c2c-8e13-62098714db5f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:33:58.143497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-79fa622c-1ef2-4c2c-8e13-62098714db5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:58.156073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa622c-1ef2-4c2c-8e13-62098714db5f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:33:58.159637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1504900-391a-4c63-b2c0-40a3b2aaffef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:33:58.160270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1504900-391a-4c63-b2c0-40a3b2aaffef service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:33:58.160551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1504900-391a-4c63-b2c0-40a3b2aaffef service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:33:58.160830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1504900-391a-4c63-b2c0-40a3b2aaffef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:33:58.162571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d1504900-391a-4c63-b2c0-40a3b2aaffef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:58.172449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1504900-391a-4c63-b2c0-40a3b2aaffef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:33:58.173858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1504900-391a-4c63-b2c0-40a3b2aaffef service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:33:58.182199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1187/2387] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:33:58.258819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977ba5e-dad7-4749-8075-fb081c7caf4d service glance] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:33:58.273213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977ba5e-dad7-4749-8075-fb081c7caf4d service glance] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:33:58.278912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8c2b2-7530-4b9c-9320-3ccacf1dd042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:33:58.279579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8c2b2-7530-4b9c-9320-3ccacf1dd042 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:33:58.279869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8c2b2-7530-4b9c-9320-3ccacf1dd042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:33:58.280153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8c2b2-7530-4b9c-9320-3ccacf1dd042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:33:58.282017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-70a8c2b2-7530-4b9c-9320-3ccacf1dd042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:33:58.296247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8c2b2-7530-4b9c-9320-3ccacf1dd042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:33:58.300487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8c2b2-7530-4b9c-9320-3ccacf1dd042 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:33:58.309530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1201/2388] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:34:01.908426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3f6fd-d73a-403b-8fdf-171f3033f17f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:01.908976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3f6fd-d73a-403b-8fdf-171f3033f17f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:01.908976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3f6fd-d73a-403b-8fdf-171f3033f17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:01.931178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f2b3f6fd-d73a-403b-8fdf-171f3033f17f None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:34:01.932989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f2b3f6fd-d73a-403b-8fdf-171f3033f17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:01.943474 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1188/2389] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 19 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:34:01.960900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8c2b2-7530-4b9c-9320-3ccacf1dd042 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:01.972729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8c2b2-7530-4b9c-9320-3ccacf1dd042 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:01.979305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bf25d-1af1-439e-8276-171040a4fd34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:01.982438 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bf25d-1af1-439e-8276-171040a4fd34 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:01.982929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bf25d-1af1-439e-8276-171040a4fd34 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:01.983601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bf25d-1af1-439e-8276-171040a4fd34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:01.998171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bf25d-1af1-439e-8276-171040a4fd34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:01.999755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bf25d-1af1-439e-8276-171040a4fd34 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:02.010101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1202/2390] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:34:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:34:11.925085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=34 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:12.090839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=66 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:17.494878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3f6fd-d73a-403b-8fdf-171f3033f17f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:17.504584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3f6fd-d73a-403b-8fdf-171f3033f17f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:17.508225 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5934b61-f714-4e67-8948-380cd60a2a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:17.508867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5934b61-f714-4e67-8948-380cd60a2a74 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:17.509153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5934b61-f714-4e67-8948-380cd60a2a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:17.509438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5934b61-f714-4e67-8948-380cd60a2a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:17.511242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e5934b61-f714-4e67-8948-380cd60a2a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:17.525313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5934b61-f714-4e67-8948-380cd60a2a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:17.526775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5934b61-f714-4e67-8948-380cd60a2a74 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:17.539004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1189/2391] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:34:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:34:18.665931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bf25d-1af1-439e-8276-171040a4fd34 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:18.682118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bf25d-1af1-439e-8276-171040a4fd34 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:18.690362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fe581-de3e-4b0a-acfc-c60ea9f578cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:18.691561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fe581-de3e-4b0a-acfc-c60ea9f578cf service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:18.695350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fe581-de3e-4b0a-acfc-c60ea9f578cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:18.695350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fe581-de3e-4b0a-acfc-c60ea9f578cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:18.695350 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-908fe581-de3e-4b0a-acfc-c60ea9f578cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:18.711094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fe581-de3e-4b0a-acfc-c60ea9f578cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:18.713060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fe581-de3e-4b0a-acfc-c60ea9f578cf service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:18.729861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1203/2392] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:34:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:34:24.319814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923561c3-90a0-44d7-b386-1fc8e11fa68e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:24.320309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923561c3-90a0-44d7-b386-1fc8e11fa68e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:24.320408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923561c3-90a0-44d7-b386-1fc8e11fa68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:24.338021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-923561c3-90a0-44d7-b386-1fc8e11fa68e None None] MFA Rules not processed for user `6be447727ed44a48a6b023303289e319`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:34:24.339261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-923561c3-90a0-44d7-b386-1fc8e11fa68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:24.347040 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1190/2393] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 19 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:34:24.359967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fe581-de3e-4b0a-acfc-c60ea9f578cf service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:24.371258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fe581-de3e-4b0a-acfc-c60ea9f578cf service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:24.376356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd44c9-4c62-4652-aad9-6e1e4b272af3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28ddf898ad7b4e10aaf0770249b4453d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:24.376991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd44c9-4c62-4652-aad9-6e1e4b272af3 service swift] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:24.377276 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd44c9-4c62-4652-aad9-6e1e4b272af3 service swift] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:24.377554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fd44c9-4c62-4652-aad9-6e1e4b272af3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:24.385621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd44c9-4c62-4652-aad9-6e1e4b272af3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:24.387246 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fd44c9-4c62-4652-aad9-6e1e4b272af3 service swift] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:24.395907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1204/2394] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 19 12:34:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 12:34:34.332718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:34.392569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=12 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:36.812937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2529b-ccf1-4c59-b86e-aa1a47e24dda None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:36.815902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2529b-ccf1-4c59-b86e-aa1a47e24dda None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:36.815902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2529b-ccf1-4c59-b86e-aa1a47e24dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:36.836568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-85d2529b-ccf1-4c59-b86e-aa1a47e24dda None None] MFA Rules not processed for user `8f609cfe0d7a4ded8a622c65d8690f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:34:36.837841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-85d2529b-ccf1-4c59-b86e-aa1a47e24dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:36.846641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1191/2395] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:34:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:34:43.146926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd44c9-4c62-4652-aad9-6e1e4b272af3 service swift] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:43.156963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fd44c9-4c62-4652-aad9-6e1e4b272af3 service swift] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:43.160649 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:43.161321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:43.161606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:43.161885 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:43.163707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:43.173946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:43.175408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:43.183624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1205/2396] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:34:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:34:46.829714 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:53.179786 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=6 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:53.231773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2529b-ccf1-4c59-b86e-aa1a47e24dda None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:53.241868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2529b-ccf1-4c59-b86e-aa1a47e24dda None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:53.245407 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e736f7-3aea-464f-adb3-bc6e060bc08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:53.246041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e736f7-3aea-464f-adb3-bc6e060bc08c service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:53.246324 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e736f7-3aea-464f-adb3-bc6e060bc08c service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:53.246713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e736f7-3aea-464f-adb3-bc6e060bc08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:53.248466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-08e736f7-3aea-464f-adb3-bc6e060bc08c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:53.263501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e736f7-3aea-464f-adb3-bc6e060bc08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:53.265641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e736f7-3aea-464f-adb3-bc6e060bc08c service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:53.276201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1192/2397] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:34:56.961857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:56.963380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:56.981205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793f60c-2546-41b5-a0d5-80e521cc9d02 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:56.986034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:56.987007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:56.987293 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:56.987609 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] PATH_INFO: `/v3/users/707a899d6c5d4c4ea1c58e0de6a85b1d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:57.001534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] RBAC: Authorizing `identity:delete_user(user_id=707a899d6c5d4c4ea1c58e0de6a85b1d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:57.003496 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:57.196677 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707a899d6c5d4c4ea1c58e0de6a85b1d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:57.207024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707a899d6c5d4c4ea1c58e0de6a85b1d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:57.207746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707a899d6c5d4c4ea1c58e0de6a85b1d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:57.208357 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707a899d6c5d4c4ea1c58e0de6a85b1d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:57.292226 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6f561987-71e5-414e-89ab-6a2ed5bc931f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707a899d6c5d4c4ea1c58e0de6a85b1d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:57.310440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1206/2398] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:34:56 2022] DELETE /identity/v3/users/707a899d6c5d4c4ea1c58e0de6a85b1d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:34:57.553638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e736f7-3aea-464f-adb3-bc6e060bc08c service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:57.565984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e736f7-3aea-464f-adb3-bc6e060bc08c service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:57.630921 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:57.631783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:57.632065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:57.632622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] PATH_INFO: `/v3/projects/4f4a6b5e282e45ffbef6963200558594` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:57.644346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f4a6b5e282e45ffbef6963200558594)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:57.646317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:57.938653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f4a6b5e282e45ffbef6963200558594', 'request_id': 'req-8c34631a-5453-403c-936a-e34996620ab0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:57.944290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f4a6b5e282e45ffbef6963200558594', 'request_id': 'req-8c34631a-5453-403c-936a-e34996620ab0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:58.045793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f4a6b5e282e45ffbef6963200558594', 'request_id': 'req-8c34631a-5453-403c-936a-e34996620ab0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:58.058593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] The token cache is being invalidate because project 4f4a6b5e282e45ffbef6963200558594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:34:58.058884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:58.059521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8c34631a-5453-403c-936a-e34996620ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:34:58.064370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1193/2399] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:34:57 2022] DELETE /identity/v3/projects/4f4a6b5e282e45ffbef6963200558594 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:34:58.066653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15375910-f68f-4ccd-aded-2c194eb8b07d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.067166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15375910-f68f-4ccd-aded-2c194eb8b07d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.067261 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15375910-f68f-4ccd-aded-2c194eb8b07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.086805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-15375910-f68f-4ccd-aded-2c194eb8b07d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:34:58.088068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-15375910-f68f-4ccd-aded-2c194eb8b07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:58.151298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1207/2400] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:34:58.159323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba5645-0059-441e-9404-ef8cc3569720 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.159619 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba5645-0059-441e-9404-ef8cc3569720 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.159906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba5645-0059-441e-9404-ef8cc3569720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.190244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-40ba5645-0059-441e-9404-ef8cc3569720 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:34:58.191417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-40ba5645-0059-441e-9404-ef8cc3569720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:58.199257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1194/2401] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:34:58.204383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15375910-f68f-4ccd-aded-2c194eb8b07d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:58.214160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15375910-f68f-4ccd-aded-2c194eb8b07d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:58.219259 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcf302-e041-4c61-863f-147ba58b44ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:58.219997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcf302-e041-4c61-863f-147ba58b44ef admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.220280 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcf302-e041-4c61-863f-147ba58b44ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.220564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcf302-e041-4c61-863f-147ba58b44ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.221317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcf302-e041-4c61-863f-147ba58b44ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:34:58.221666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcf302-e041-4c61-863f-147ba58b44ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:58.223366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcf302-e041-4c61-863f-147ba58b44ef admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:58.235795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1208/2402] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:34:58.240148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba5645-0059-441e-9404-ef8cc3569720 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:58.249261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba5645-0059-441e-9404-ef8cc3569720 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:58.254187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c17d4-bc02-4963-80d7-635a784487fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:58.254862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c17d4-bc02-4963-80d7-635a784487fd admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.255144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c17d4-bc02-4963-80d7-635a784487fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.255417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c17d4-bc02-4963-80d7-635a784487fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.256113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c17d4-bc02-4963-80d7-635a784487fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:58.257992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c17d4-bc02-4963-80d7-635a784487fd admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:58.355996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1195/2403] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:34:58 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:34:58.360699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcf302-e041-4c61-863f-147ba58b44ef admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:58.370716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcf302-e041-4c61-863f-147ba58b44ef admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:58.416270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:58.427989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831298ba-d2a8-4266-8d49-96e60b5c7910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:58.428655 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831298ba-d2a8-4266-8d49-96e60b5c7910 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.428941 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831298ba-d2a8-4266-8d49-96e60b5c7910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.432066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831298ba-d2a8-4266-8d49-96e60b5c7910 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.432066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831298ba-d2a8-4266-8d49-96e60b5c7910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:58.432266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831298ba-d2a8-4266-8d49-96e60b5c7910 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:58.555761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1209/2404] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:34:58 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:34:58.559917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c17d4-bc02-4963-80d7-635a784487fd admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:58.570507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c17d4-bc02-4963-80d7-635a784487fd admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:58.574932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3e434-0cd3-43da-bbd7-3ed2b5c274bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:58.575593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3e434-0cd3-43da-bbd7-3ed2b5c274bf admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.575967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3e434-0cd3-43da-bbd7-3ed2b5c274bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.576259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3e434-0cd3-43da-bbd7-3ed2b5c274bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.576976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3e434-0cd3-43da-bbd7-3ed2b5c274bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:58.579250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3e434-0cd3-43da-bbd7-3ed2b5c274bf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:58.589412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1196/2405] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:34:58.594798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831298ba-d2a8-4266-8d49-96e60b5c7910 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:58.604971 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831298ba-d2a8-4266-8d49-96e60b5c7910 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:58.610056 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a2646-e6b3-4dca-8e53-47a3f9d83d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:58.610759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a2646-e6b3-4dca-8e53-47a3f9d83d0a admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.611039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a2646-e6b3-4dca-8e53-47a3f9d83d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.611341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a2646-e6b3-4dca-8e53-47a3f9d83d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.612060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a2646-e6b3-4dca-8e53-47a3f9d83d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:58.613663 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a2646-e6b3-4dca-8e53-47a3f9d83d0a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:58.623681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1210/2406] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:34:58.628234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3e434-0cd3-43da-bbd7-3ed2b5c274bf admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:58.638012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3e434-0cd3-43da-bbd7-3ed2b5c274bf admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:58.643507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a80968-6371-4b9e-85d3-af85aab077d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:58.644479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a80968-6371-4b9e-85d3-af85aab077d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.644760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a80968-6371-4b9e-85d3-af85aab077d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.645039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a80968-6371-4b9e-85d3-af85aab077d1 admin admin] PATH_INFO: `/v3/projects/b1ea2ef2f62242afa600281ef3b3da6e/users/f88ae0fa9e8c4f568ecef7524d312032/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.661251 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a80968-6371-4b9e-85d3-af85aab077d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ea2ef2f62242afa600281ef3b3da6e, user_id=f88ae0fa9e8c4f568ecef7524d312032, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:58.666077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a80968-6371-4b9e-85d3-af85aab077d1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:58.750831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1197/2407] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:34:58 2022] PUT /identity/v3/projects/b1ea2ef2f62242afa600281ef3b3da6e/users/f88ae0fa9e8c4f568ecef7524d312032/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:34:58.762058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4450435-6f12-473e-b8b9-a15620b33af4 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.762348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4450435-6f12-473e-b8b9-a15620b33af4 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.762693 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4450435-6f12-473e-b8b9-a15620b33af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.790965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d4450435-6f12-473e-b8b9-a15620b33af4 None None] MFA Rules not processed for user `f88ae0fa9e8c4f568ecef7524d312032`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:34:58.792584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d4450435-6f12-473e-b8b9-a15620b33af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:58.862585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1211/2408] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:34:58.867035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a80968-6371-4b9e-85d3-af85aab077d1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:58.872765 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:58.928564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a80968-6371-4b9e-85d3-af85aab077d1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:58.933954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d870147-0e5a-4925-86e9-b8cbb8b9d7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:58.934632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d870147-0e5a-4925-86e9-b8cbb8b9d7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:58.934917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d870147-0e5a-4925-86e9-b8cbb8b9d7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:58.935212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d870147-0e5a-4925-86e9-b8cbb8b9d7b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:58.935896 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d870147-0e5a-4925-86e9-b8cbb8b9d7b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:58.937928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d870147-0e5a-4925-86e9-b8cbb8b9d7b5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:59.064729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1198/2409] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:34:58 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:34:59.069050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4450435-6f12-473e-b8b9-a15620b33af4 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:59.080473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4450435-6f12-473e-b8b9-a15620b33af4 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:59.136308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2f0ee-c984-48cd-aa3f-865c570df37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:59.136981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2f0ee-c984-48cd-aa3f-865c570df37d admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:59.137291 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2f0ee-c984-48cd-aa3f-865c570df37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:59.137570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2f0ee-c984-48cd-aa3f-865c570df37d admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:59.138273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2f0ee-c984-48cd-aa3f-865c570df37d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:59.140898 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2f0ee-c984-48cd-aa3f-865c570df37d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:59.258202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1212/2410] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:34:59 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:34:59.262275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d870147-0e5a-4925-86e9-b8cbb8b9d7b5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:59.274746 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d870147-0e5a-4925-86e9-b8cbb8b9d7b5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:59.278349 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4df5-3b4e-4e04-8282-46df98dc64d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:59.279083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4df5-3b4e-4e04-8282-46df98dc64d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:59.279363 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4df5-3b4e-4e04-8282-46df98dc64d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:59.279635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4df5-3b4e-4e04-8282-46df98dc64d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:59.281793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea4df5-3b4e-4e04-8282-46df98dc64d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:59.283559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea4df5-3b4e-4e04-8282-46df98dc64d7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:59.295308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1199/2411] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:34:59.299785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2f0ee-c984-48cd-aa3f-865c570df37d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:59.309952 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2f0ee-c984-48cd-aa3f-865c570df37d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:59.315300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ebbda-b23c-44a7-8c49-587b99318fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:59.316171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ebbda-b23c-44a7-8c49-587b99318fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:59.316457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ebbda-b23c-44a7-8c49-587b99318fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:59.316735 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ebbda-b23c-44a7-8c49-587b99318fd1 admin admin] PATH_INFO: `/v3/projects/78fd26dc2e424ecd8751ab144e2fa1fd/users/6cd68d39d2ef42a6b82466f71e2c0c74/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:59.332336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ebbda-b23c-44a7-8c49-587b99318fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78fd26dc2e424ecd8751ab144e2fa1fd, user_id=6cd68d39d2ef42a6b82466f71e2c0c74, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:59.335038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ebbda-b23c-44a7-8c49-587b99318fd1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:59.429563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1213/2412] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:34:59 2022] PUT /identity/v3/projects/78fd26dc2e424ecd8751ab144e2fa1fd/users/6cd68d39d2ef42a6b82466f71e2c0c74/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:34:59.433848 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4df5-3b4e-4e04-8282-46df98dc64d7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:59.443828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4df5-3b4e-4e04-8282-46df98dc64d7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:59.500462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b479c-9b70-40d8-91a3-a20004b41e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:59.501165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b479c-9b70-40d8-91a3-a20004b41e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:59.501449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b479c-9b70-40d8-91a3-a20004b41e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:59.501726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b479c-9b70-40d8-91a3-a20004b41e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:59.502494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b479c-9b70-40d8-91a3-a20004b41e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:59.504088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b479c-9b70-40d8-91a3-a20004b41e1d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:59.513957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1200/2413] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:34:59.518900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ebbda-b23c-44a7-8c49-587b99318fd1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:59.529540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ebbda-b23c-44a7-8c49-587b99318fd1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:59.534747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086614ca-fe19-4b37-92e0-0f5ad655480f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:59.535625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086614ca-fe19-4b37-92e0-0f5ad655480f admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:59.535904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086614ca-fe19-4b37-92e0-0f5ad655480f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:59.536195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086614ca-fe19-4b37-92e0-0f5ad655480f admin admin] PATH_INFO: `/v3/projects/78fd26dc2e424ecd8751ab144e2fa1fd/users/6cd68d39d2ef42a6b82466f71e2c0c74/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:59.538126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086614ca-fe19-4b37-92e0-0f5ad655480f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78fd26dc2e424ecd8751ab144e2fa1fd, user_id=6cd68d39d2ef42a6b82466f71e2c0c74, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:59.540776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086614ca-fe19-4b37-92e0-0f5ad655480f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:59.552301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1214/2414] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:34:59 2022] PUT /identity/v3/projects/78fd26dc2e424ecd8751ab144e2fa1fd/users/6cd68d39d2ef42a6b82466f71e2c0c74/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:34:59.563217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b088e2-2fb2-48e4-b6b9-737d2e8bbc0a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:59.563705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b088e2-2fb2-48e4-b6b9-737d2e8bbc0a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:59.563800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b088e2-2fb2-48e4-b6b9-737d2e8bbc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:59.590733 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-c7b088e2-2fb2-48e4-b6b9-737d2e8bbc0a None None] MFA Rules not processed for user `6cd68d39d2ef42a6b82466f71e2c0c74`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:34:59.591937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c7b088e2-2fb2-48e4-b6b9-737d2e8bbc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:59.657527 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:59.661076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1201/2415] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:34:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:34:59.672828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086614ca-fe19-4b37-92e0-0f5ad655480f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:59.674594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-086614ca-fe19-4b37-92e0-0f5ad655480f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:34:59.685893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:34:59.732918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086614ca-fe19-4b37-92e0-0f5ad655480f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:59.736740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00c69e-0cb4-4689-ad39-d1f3599f44b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:59.737384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00c69e-0cb4-4689-ad39-d1f3599f44b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:59.737668 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00c69e-0cb4-4689-ad39-d1f3599f44b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:59.737943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00c69e-0cb4-4689-ad39-d1f3599f44b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:34:59.746975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00c69e-0cb4-4689-ad39-d1f3599f44b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:34:59.748409 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00c69e-0cb4-4689-ad39-d1f3599f44b8 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:34:59.758342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1215/2416] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:34:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:34:59.973443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b088e2-2fb2-48e4-b6b9-737d2e8bbc0a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:34:59.983660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b088e2-2fb2-48e4-b6b9-737d2e8bbc0a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:34:59.988051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3796361-cbd0-4107-9271-ba5af18f450b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:34:59.988694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796361-cbd0-4107-9271-ba5af18f450b service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:34:59.988979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796361-cbd0-4107-9271-ba5af18f450b service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:34:59.989496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3796361-cbd0-4107-9271-ba5af18f450b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:00.064159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3796361-cbd0-4107-9271-ba5af18f450b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:00.065601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3796361-cbd0-4107-9271-ba5af18f450b service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:00.074766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1202/2417] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:34:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:35:09.857247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=22 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:10.070610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=28 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:21.904961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00c69e-0cb4-4689-ad39-d1f3599f44b8 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:21.966344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00c69e-0cb4-4689-ad39-d1f3599f44b8 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:21.970844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:21.971798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:21.972075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:21.972347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] PATH_INFO: `/v3/users/f88ae0fa9e8c4f568ecef7524d312032` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:21.973548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f88ae0fa9e8c4f568ecef7524d312032)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:21.975513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:22.149939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f88ae0fa9e8c4f568ecef7524d312032'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:22.158863 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f88ae0fa9e8c4f568ecef7524d312032'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:22.159593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f88ae0fa9e8c4f568ecef7524d312032'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:22.160442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f88ae0fa9e8c4f568ecef7524d312032'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:22.216868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f88ae0fa9e8c4f568ecef7524d312032'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:22.236489 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1216/2418] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:35:21 2022] DELETE /identity/v3/users/f88ae0fa9e8c4f568ecef7524d312032 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:22.241029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e5ab6-8409-45a2-8098-235baaa10dd0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:22.242165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e5ab6-8409-45a2-8098-235baaa10dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:22.242279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e5ab6-8409-45a2-8098-235baaa10dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:22.259106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-f46e5ab6-8409-45a2-8098-235baaa10dd0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:22.260307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f46e5ab6-8409-45a2-8098-235baaa10dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:22.320295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1203/2419] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:22.330011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:22.331433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:22.389871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f02e8-a163-4fd5-b514-53700a63fdf6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:22.393534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e549b-afee-4b46-bb56-810d32874767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:22.394331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e549b-afee-4b46-bb56-810d32874767 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:22.394494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e549b-afee-4b46-bb56-810d32874767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:22.394775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e549b-afee-4b46-bb56-810d32874767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:22.408540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e549b-afee-4b46-bb56-810d32874767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:22.408540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e549b-afee-4b46-bb56-810d32874767 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:22.420694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1217/2420] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:35:22.905055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e5ab6-8409-45a2-8098-235baaa10dd0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:22.906594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f46e5ab6-8409-45a2-8098-235baaa10dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:22.948108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e5ab6-8409-45a2-8098-235baaa10dd0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:22.952936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:22.953800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:22.954085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:22.954383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] PATH_INFO: `/v3/projects/b1ea2ef2f62242afa600281ef3b3da6e` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:22.955687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ea2ef2f62242afa600281ef3b3da6e)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:22.958119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:23.114605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ea2ef2f62242afa600281ef3b3da6e', 'request_id': 'req-c6b75715-4177-46a8-bc55-410dfc27ff0a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.120113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ea2ef2f62242afa600281ef3b3da6e', 'request_id': 'req-c6b75715-4177-46a8-bc55-410dfc27ff0a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.193912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ea2ef2f62242afa600281ef3b3da6e', 'request_id': 'req-c6b75715-4177-46a8-bc55-410dfc27ff0a'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.206554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] The token cache is being invalidate because project b1ea2ef2f62242afa600281ef3b3da6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:35:23.206843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.207478 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.209008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1204/2421] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:35:22 2022] DELETE /identity/v3/projects/b1ea2ef2f62242afa600281ef3b3da6e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:23.213603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e549b-afee-4b46-bb56-810d32874767 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:23.238954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:23.247076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e549b-afee-4b46-bb56-810d32874767 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:23.306786 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:23.307665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:23.307947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:23.308221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] PATH_INFO: `/v3/users/6cd68d39d2ef42a6b82466f71e2c0c74` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:23.309414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cd68d39d2ef42a6b82466f71e2c0c74)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:23.311434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:23.581470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cd68d39d2ef42a6b82466f71e2c0c74'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.588790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cd68d39d2ef42a6b82466f71e2c0c74'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.589471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cd68d39d2ef42a6b82466f71e2c0c74'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.590080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cd68d39d2ef42a6b82466f71e2c0c74'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.703144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-162cdc7e-cd96-46e2-a3d9-065c71e509a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cd68d39d2ef42a6b82466f71e2c0c74'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:23.723297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1218/2422] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:35:23 2022] DELETE /identity/v3/users/6cd68d39d2ef42a6b82466f71e2c0c74 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:24.282733 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:24.284363 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:24.328107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b75715-4177-46a8-bc55-410dfc27ff0a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:24.367201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:24.387360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:24.388235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:24.388522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:24.388798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] PATH_INFO: `/v3/projects/78fd26dc2e424ecd8751ab144e2fa1fd` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:24.389987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=78fd26dc2e424ecd8751ab144e2fa1fd)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:24.391958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:24.536669 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78fd26dc2e424ecd8751ab144e2fa1fd', 'request_id': 'req-4c4f6e64-46e6-4881-977e-cc454b6879cb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:24.543586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78fd26dc2e424ecd8751ab144e2fa1fd', 'request_id': 'req-4c4f6e64-46e6-4881-977e-cc454b6879cb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:24.685540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78fd26dc2e424ecd8751ab144e2fa1fd', 'request_id': 'req-4c4f6e64-46e6-4881-977e-cc454b6879cb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:24.698307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] The token cache is being invalidate because project 78fd26dc2e424ecd8751ab144e2fa1fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:35:24.698668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:24.699262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4c4f6e64-46e6-4881-977e-cc454b6879cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:24.700828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1205/2423] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:35:24 2022] DELETE /identity/v3/projects/78fd26dc2e424ecd8751ab144e2fa1fd => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:24.705743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072632a-22a5-4311-9c47-c8dbd4b1abe8 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:24.706278 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072632a-22a5-4311-9c47-c8dbd4b1abe8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:24.706381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072632a-22a5-4311-9c47-c8dbd4b1abe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:24.744818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c072632a-22a5-4311-9c47-c8dbd4b1abe8 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:24.746034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c072632a-22a5-4311-9c47-c8dbd4b1abe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:24.817177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1219/2424] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:24.826776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886fa7a-6854-4193-8c6a-1dca9405bbe1 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:24.827065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886fa7a-6854-4193-8c6a-1dca9405bbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:24.827377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886fa7a-6854-4193-8c6a-1dca9405bbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:24.846361 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9886fa7a-6854-4193-8c6a-1dca9405bbe1 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:24.847608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9886fa7a-6854-4193-8c6a-1dca9405bbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:24.855629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1206/2425] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:24.860455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072632a-22a5-4311-9c47-c8dbd4b1abe8 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:24.866992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:24.871071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072632a-22a5-4311-9c47-c8dbd4b1abe8 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:24.875554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47549088-6d26-4e12-ac92-79149af92d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:24.880051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47549088-6d26-4e12-ac92-79149af92d31 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:24.880051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47549088-6d26-4e12-ac92-79149af92d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:24.880051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47549088-6d26-4e12-ac92-79149af92d31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:24.880051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47549088-6d26-4e12-ac92-79149af92d31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:35:24.880051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47549088-6d26-4e12-ac92-79149af92d31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:24.880051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47549088-6d26-4e12-ac92-79149af92d31 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:24.892369 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1220/2426] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:35:24.896696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886fa7a-6854-4193-8c6a-1dca9405bbe1 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:24.906520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886fa7a-6854-4193-8c6a-1dca9405bbe1 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:24.911191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aefa51-d5aa-4946-b0f3-544643d823b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:24.914482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aefa51-d5aa-4946-b0f3-544643d823b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:24.914482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aefa51-d5aa-4946-b0f3-544643d823b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:24.914482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aefa51-d5aa-4946-b0f3-544643d823b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:24.914482 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aefa51-d5aa-4946-b0f3-544643d823b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:24.915456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aefa51-d5aa-4946-b0f3-544643d823b9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:25.020853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1207/2427] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:35:24 2022] POST /identity/v3/projects => generated 363 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:35:25.023458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47549088-6d26-4e12-ac92-79149af92d31 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:25.034266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47549088-6d26-4e12-ac92-79149af92d31 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:25.091559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f080fb-a6ce-4166-b6b7-fd62f21e6265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:25.092138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f080fb-a6ce-4166-b6b7-fd62f21e6265 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:25.092421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f080fb-a6ce-4166-b6b7-fd62f21e6265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:25.092697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f080fb-a6ce-4166-b6b7-fd62f21e6265 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:25.093388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f080fb-a6ce-4166-b6b7-fd62f21e6265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:25.095332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f080fb-a6ce-4166-b6b7-fd62f21e6265 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:25.223985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1221/2428] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:35:25 2022] POST /identity/v3/users => generated 282 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:25.229129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aefa51-d5aa-4946-b0f3-544643d823b9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:25.239657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aefa51-d5aa-4946-b0f3-544643d823b9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:25.244987 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763001a-1c4d-48a7-a59a-f582624e3105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:25.248261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763001a-1c4d-48a7-a59a-f582624e3105 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:25.248261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763001a-1c4d-48a7-a59a-f582624e3105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:25.248261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763001a-1c4d-48a7-a59a-f582624e3105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:25.248261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763001a-1c4d-48a7-a59a-f582624e3105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:25.248617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763001a-1c4d-48a7-a59a-f582624e3105 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:25.260801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1208/2429] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:25.265052 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f080fb-a6ce-4166-b6b7-fd62f21e6265 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:25.275692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f080fb-a6ce-4166-b6b7-fd62f21e6265 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:25.280569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f99f6d-9d6e-411d-bb0f-b527275ab632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:25.284083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f99f6d-9d6e-411d-bb0f-b527275ab632 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:25.284083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f99f6d-9d6e-411d-bb0f-b527275ab632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:25.284083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f99f6d-9d6e-411d-bb0f-b527275ab632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:25.284083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f99f6d-9d6e-411d-bb0f-b527275ab632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:25.284563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f99f6d-9d6e-411d-bb0f-b527275ab632 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:25.295353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1222/2430] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:25.299674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763001a-1c4d-48a7-a59a-f582624e3105 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:25.311283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763001a-1c4d-48a7-a59a-f582624e3105 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:25.316103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab2087-eeff-467e-88e8-4976d7407ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:25.316990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab2087-eeff-467e-88e8-4976d7407ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:25.317275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab2087-eeff-467e-88e8-4976d7407ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:25.317549 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab2087-eeff-467e-88e8-4976d7407ae1 admin admin] PATH_INFO: `/v3/projects/b905757e8e06496aaffbbef6e843d07d/users/2f0da68c3247409d85cf58fbaf01dbe9/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:25.333989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab2087-eeff-467e-88e8-4976d7407ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b905757e8e06496aaffbbef6e843d07d, user_id=2f0da68c3247409d85cf58fbaf01dbe9, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:25.336809 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab2087-eeff-467e-88e8-4976d7407ae1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:25.411685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1209/2431] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:35:25 2022] PUT /identity/v3/projects/b905757e8e06496aaffbbef6e843d07d/users/2f0da68c3247409d85cf58fbaf01dbe9/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:25.422117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe045d2-0a05-4180-aff0-90d30ac31cdb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:25.422513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe045d2-0a05-4180-aff0-90d30ac31cdb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:25.422926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe045d2-0a05-4180-aff0-90d30ac31cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:25.450303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-bfe045d2-0a05-4180-aff0-90d30ac31cdb None None] MFA Rules not processed for user `2f0da68c3247409d85cf58fbaf01dbe9`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:25.451680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bfe045d2-0a05-4180-aff0-90d30ac31cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:25.524845 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1223/2432] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:25 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:35:25.529355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab2087-eeff-467e-88e8-4976d7407ae1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:25.595103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab2087-eeff-467e-88e8-4976d7407ae1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:25.599807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45564416-b498-4434-967c-4071139a36cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:25.600469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45564416-b498-4434-967c-4071139a36cf admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:25.601235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45564416-b498-4434-967c-4071139a36cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:25.601535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45564416-b498-4434-967c-4071139a36cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:25.602232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45564416-b498-4434-967c-4071139a36cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:25.604167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45564416-b498-4434-967c-4071139a36cf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:25.614016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:25.721902 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1210/2433] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:35:25 2022] POST /identity/v3/projects => generated 363 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:25.726431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe045d2-0a05-4180-aff0-90d30ac31cdb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:25.736992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe045d2-0a05-4180-aff0-90d30ac31cdb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:25.795579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ae89f-7dfe-43a3-a58d-7d90ed78147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:25.796284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae89f-7dfe-43a3-a58d-7d90ed78147e admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:25.796568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae89f-7dfe-43a3-a58d-7d90ed78147e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:25.796846 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae89f-7dfe-43a3-a58d-7d90ed78147e admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:25.797541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ae89f-7dfe-43a3-a58d-7d90ed78147e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:25.799585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ae89f-7dfe-43a3-a58d-7d90ed78147e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:25.807189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:25.914475 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1224/2434] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:35:25 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:25.918574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45564416-b498-4434-967c-4071139a36cf admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:25.929399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45564416-b498-4434-967c-4071139a36cf admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:25.934699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c2a6-f3c5-4426-91a6-f37ae37b81d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:25.935357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c2a6-f3c5-4426-91a6-f37ae37b81d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:25.935640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c2a6-f3c5-4426-91a6-f37ae37b81d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:25.935917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c2a6-f3c5-4426-91a6-f37ae37b81d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:25.936632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c2a6-f3c5-4426-91a6-f37ae37b81d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:25.938294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c2a6-f3c5-4426-91a6-f37ae37b81d5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:25.949484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1211/2435] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:25.953590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ae89f-7dfe-43a3-a58d-7d90ed78147e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:25.964050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ae89f-7dfe-43a3-a58d-7d90ed78147e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:25.968739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a1e33-68e9-4471-80e8-5da6492fb51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:25.969628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a1e33-68e9-4471-80e8-5da6492fb51f admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:25.969914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a1e33-68e9-4471-80e8-5da6492fb51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:25.970196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a1e33-68e9-4471-80e8-5da6492fb51f admin admin] PATH_INFO: `/v3/projects/7e43ed2b77d148d7bde76d4fa0dce317/users/f683afc6fadb4bdbb5dbafb9bd328652/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:25.985785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a1e33-68e9-4471-80e8-5da6492fb51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e43ed2b77d148d7bde76d4fa0dce317, user_id=f683afc6fadb4bdbb5dbafb9bd328652, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:25.988513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a1e33-68e9-4471-80e8-5da6492fb51f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:26.104543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1225/2436] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:35:25 2022] PUT /identity/v3/projects/7e43ed2b77d148d7bde76d4fa0dce317/users/f683afc6fadb4bdbb5dbafb9bd328652/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:26.108720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c2a6-f3c5-4426-91a6-f37ae37b81d5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:26.119588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c2a6-f3c5-4426-91a6-f37ae37b81d5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:26.175595 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ff354-9029-485d-b50e-b0ba4c274082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:26.176173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ff354-9029-485d-b50e-b0ba4c274082 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:26.176460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ff354-9029-485d-b50e-b0ba4c274082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:26.176741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ff354-9029-485d-b50e-b0ba4c274082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:26.177465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ff354-9029-485d-b50e-b0ba4c274082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:26.179678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ff354-9029-485d-b50e-b0ba4c274082 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:26.190030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1212/2437] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:26.194271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a1e33-68e9-4471-80e8-5da6492fb51f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:26.204593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a1e33-68e9-4471-80e8-5da6492fb51f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:26.209274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a25a-c157-45b9-9202-79de49eaeb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:26.210175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a25a-c157-45b9-9202-79de49eaeb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:26.210490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a25a-c157-45b9-9202-79de49eaeb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:26.210764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a25a-c157-45b9-9202-79de49eaeb15 admin admin] PATH_INFO: `/v3/projects/7e43ed2b77d148d7bde76d4fa0dce317/users/f683afc6fadb4bdbb5dbafb9bd328652/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:26.212719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647a25a-c157-45b9-9202-79de49eaeb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e43ed2b77d148d7bde76d4fa0dce317, user_id=f683afc6fadb4bdbb5dbafb9bd328652, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:26.215385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647a25a-c157-45b9-9202-79de49eaeb15 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:26.226955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1226/2438] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:35:26 2022] PUT /identity/v3/projects/7e43ed2b77d148d7bde76d4fa0dce317/users/f683afc6fadb4bdbb5dbafb9bd328652/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:26.237426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b6aac-8902-4fab-9a7f-72e10ecac7bf None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:26.237797 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b6aac-8902-4fab-9a7f-72e10ecac7bf None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:26.238090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b6aac-8902-4fab-9a7f-72e10ecac7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:26.266841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5d6b6aac-8902-4fab-9a7f-72e10ecac7bf None None] MFA Rules not processed for user `f683afc6fadb4bdbb5dbafb9bd328652`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:26.268141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5d6b6aac-8902-4fab-9a7f-72e10ecac7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:26.340393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1213/2439] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:26 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:26.352724 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a25a-c157-45b9-9202-79de49eaeb15 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:26.354656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0647a25a-c157-45b9-9202-79de49eaeb15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:26.429353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a25a-c157-45b9-9202-79de49eaeb15 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:26.433266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f1a08-8d81-4ea8-bb80-07f0ca8b9a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:26.433912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1a08-8d81-4ea8-bb80-07f0ca8b9a37 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:26.434199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1a08-8d81-4ea8-bb80-07f0ca8b9a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:26.434511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1a08-8d81-4ea8-bb80-07f0ca8b9a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:26.443810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f1a08-8d81-4ea8-bb80-07f0ca8b9a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:26.445391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f1a08-8d81-4ea8-bb80-07f0ca8b9a37 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:26.454987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1227/2440] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:26 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:35:26.630660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b6aac-8902-4fab-9a7f-72e10ecac7bf None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:26.640627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b6aac-8902-4fab-9a7f-72e10ecac7bf None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:26.644198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9253f-782a-4598-a7d8-12e652ef21de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:26.644845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9253f-782a-4598-a7d8-12e652ef21de service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:26.645133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9253f-782a-4598-a7d8-12e652ef21de service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:26.645977 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9253f-782a-4598-a7d8-12e652ef21de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:26.711860 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9253f-782a-4598-a7d8-12e652ef21de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:26.713310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9253f-782a-4598-a7d8-12e652ef21de service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:26.719079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:26.727374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1214/2441] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:35:27.149504 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f1a08-8d81-4ea8-bb80-07f0ca8b9a37 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:27.152426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-087f1a08-8d81-4ea8-bb80-07f0ca8b9a37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:27.227334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f1a08-8d81-4ea8-bb80-07f0ca8b9a37 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:27.232759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3525cc-b853-4681-b3bf-182836c0091a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:27.233410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3525cc-b853-4681-b3bf-182836c0091a service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:27.233696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3525cc-b853-4681-b3bf-182836c0091a service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:27.233974 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3525cc-b853-4681-b3bf-182836c0091a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:27.235798 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fa3525cc-b853-4681-b3bf-182836c0091a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:27.248868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3525cc-b853-4681-b3bf-182836c0091a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:27.250611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3525cc-b853-4681-b3bf-182836c0091a service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:27.261434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1228/2442] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:35:37.257180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:INSERT=5,keystone:SELECT=84 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:44.408711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9253f-782a-4598-a7d8-12e652ef21de service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:44.490253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9253f-782a-4598-a7d8-12e652ef21de service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:44.495627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:44.496493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:44.496776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:44.497053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] PATH_INFO: `/v3/users/2f0da68c3247409d85cf58fbaf01dbe9` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:44.498246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f0da68c3247409d85cf58fbaf01dbe9)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:44.500217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:44.705821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f0da68c3247409d85cf58fbaf01dbe9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:44.706533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f0da68c3247409d85cf58fbaf01dbe9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:44.719157 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f0da68c3247409d85cf58fbaf01dbe9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:44.719801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f0da68c3247409d85cf58fbaf01dbe9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:44.816164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f0da68c3247409d85cf58fbaf01dbe9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:44.829499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1215/2443] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:35:44 2022] DELETE /identity/v3/users/2f0da68c3247409d85cf58fbaf01dbe9 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:44.834132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313ef78-ad94-4add-a1a1-45ecd2ea1fe8 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:44.834711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313ef78-ad94-4add-a1a1-45ecd2ea1fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:44.834826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313ef78-ad94-4add-a1a1-45ecd2ea1fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:44.853149 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6313ef78-ad94-4add-a1a1-45ecd2ea1fe8 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:44.854342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6313ef78-ad94-4add-a1a1-45ecd2ea1fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:44.917083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1229/2444] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:44.928180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:44.930671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:45.004035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:45.013205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbacb2-6884-4c8b-88a7-227c7ea9c93e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:45.017002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5e09c-4bce-4393-8167-4b84086a8c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:45.017699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5e09c-4bce-4393-8167-4b84086a8c27 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:45.017988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5e09c-4bce-4393-8167-4b84086a8c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:45.018264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5e09c-4bce-4393-8167-4b84086a8c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:45.028581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5e09c-4bce-4393-8167-4b84086a8c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:45.030010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5e09c-4bce-4393-8167-4b84086a8c27 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:45.040296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1216/2445] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:35:45.492141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313ef78-ad94-4add-a1a1-45ecd2ea1fe8 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:45.493576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6313ef78-ad94-4add-a1a1-45ecd2ea1fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:45.505630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313ef78-ad94-4add-a1a1-45ecd2ea1fe8 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:45.525692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:45.525692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:45.525692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:45.525692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] PATH_INFO: `/v3/projects/b905757e8e06496aaffbbef6e843d07d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:45.526608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b905757e8e06496aaffbbef6e843d07d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:45.526608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:45.652537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b905757e8e06496aaffbbef6e843d07d', 'request_id': 'req-91386424-e107-493c-af80-d7e3cd98b619'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:45.658070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b905757e8e06496aaffbbef6e843d07d', 'request_id': 'req-91386424-e107-493c-af80-d7e3cd98b619'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:45.742264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b905757e8e06496aaffbbef6e843d07d', 'request_id': 'req-91386424-e107-493c-af80-d7e3cd98b619'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:45.753989 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] The token cache is being invalidate because project b905757e8e06496aaffbbef6e843d07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:35:45.754289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:45.755055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:45.757148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1230/2446] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:35:45 2022] DELETE /identity/v3/projects/b905757e8e06496aaffbbef6e843d07d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:45.761253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5e09c-4bce-4393-8167-4b84086a8c27 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:45.771438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5e09c-4bce-4393-8167-4b84086a8c27 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:45.829706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:45.830620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:45.830909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:45.831187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] PATH_INFO: `/v3/users/f683afc6fadb4bdbb5dbafb9bd328652` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:45.832372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f683afc6fadb4bdbb5dbafb9bd328652)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:45.834303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:46.019307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f683afc6fadb4bdbb5dbafb9bd328652'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.022830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f683afc6fadb4bdbb5dbafb9bd328652'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.035904 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f683afc6fadb4bdbb5dbafb9bd328652'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.036604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f683afc6fadb4bdbb5dbafb9bd328652'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.136307 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3a41f180-8a2a-4756-86f6-3f34e9cbc161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f683afc6fadb4bdbb5dbafb9bd328652'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.149575 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1217/2447] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:35:45 2022] DELETE /identity/v3/users/f683afc6fadb4bdbb5dbafb9bd328652 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:46.583327 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:46.588289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:46.599750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91386424-e107-493c-af80-d7e3cd98b619 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:46.658400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:46.659294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:46.659583 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:46.659870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] PATH_INFO: `/v3/projects/7e43ed2b77d148d7bde76d4fa0dce317` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:46.661080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e43ed2b77d148d7bde76d4fa0dce317)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:46.663748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:46.749617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:46.767570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e43ed2b77d148d7bde76d4fa0dce317', 'request_id': 'req-6c419b7f-e167-4a72-b84c-a3e7f2772801'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.774477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e43ed2b77d148d7bde76d4fa0dce317', 'request_id': 'req-6c419b7f-e167-4a72-b84c-a3e7f2772801'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.844892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e43ed2b77d148d7bde76d4fa0dce317', 'request_id': 'req-6c419b7f-e167-4a72-b84c-a3e7f2772801'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.857122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] The token cache is being invalidate because project 7e43ed2b77d148d7bde76d4fa0dce317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:35:46.857412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.858054 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-6c419b7f-e167-4a72-b84c-a3e7f2772801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:46.859625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1231/2448] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:35:46 2022] DELETE /identity/v3/projects/7e43ed2b77d148d7bde76d4fa0dce317 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:46.864759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40cc43-a844-418f-bd1e-ba6201af1302 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:46.865223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40cc43-a844-418f-bd1e-ba6201af1302 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:46.865357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40cc43-a844-418f-bd1e-ba6201af1302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:46.904512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2c40cc43-a844-418f-bd1e-ba6201af1302 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:46.905712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2c40cc43-a844-418f-bd1e-ba6201af1302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:46.948061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:46.965638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1218/2449] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:46.973624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad956d-42e3-41e7-a2ab-5763ff006812 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:46.973913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad956d-42e3-41e7-a2ab-5763ff006812 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:46.974199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad956d-42e3-41e7-a2ab-5763ff006812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:46.992457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-57ad956d-42e3-41e7-a2ab-5763ff006812 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:46.993607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-57ad956d-42e3-41e7-a2ab-5763ff006812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:47.001480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1232/2450] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:47.005933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40cc43-a844-418f-bd1e-ba6201af1302 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.016590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40cc43-a844-418f-bd1e-ba6201af1302 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.021886 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13059851-6da5-4c9b-a892-7d2e73b296f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.022855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13059851-6da5-4c9b-a892-7d2e73b296f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.023842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13059851-6da5-4c9b-a892-7d2e73b296f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.024195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13059851-6da5-4c9b-a892-7d2e73b296f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.024994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13059851-6da5-4c9b-a892-7d2e73b296f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:35:47.025309 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13059851-6da5-4c9b-a892-7d2e73b296f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:47.026945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13059851-6da5-4c9b-a892-7d2e73b296f4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:47.039397 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1219/2451] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:35:47.043758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad956d-42e3-41e7-a2ab-5763ff006812 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.053042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad956d-42e3-41e7-a2ab-5763ff006812 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.058110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff04e9-f411-47cc-b245-17681c1d6504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.058799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff04e9-f411-47cc-b245-17681c1d6504 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.059079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff04e9-f411-47cc-b245-17681c1d6504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.059351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff04e9-f411-47cc-b245-17681c1d6504 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.060034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff04e9-f411-47cc-b245-17681c1d6504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:47.061925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff04e9-f411-47cc-b245-17681c1d6504 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:47.143303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1233/2452] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:35:47 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:47.148149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13059851-6da5-4c9b-a892-7d2e73b296f4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.158503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13059851-6da5-4c9b-a892-7d2e73b296f4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.217044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6d7a0-5e6f-41b9-8467-7043d4379ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.217753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6d7a0-5e6f-41b9-8467-7043d4379ecc admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.218009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6d7a0-5e6f-41b9-8467-7043d4379ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.218291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6d7a0-5e6f-41b9-8467-7043d4379ecc admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.219031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6d7a0-5e6f-41b9-8467-7043d4379ecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:47.220960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6d7a0-5e6f-41b9-8467-7043d4379ecc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:47.349356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1220/2453] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:35:47 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:35:47.354213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff04e9-f411-47cc-b245-17681c1d6504 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.365150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff04e9-f411-47cc-b245-17681c1d6504 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.370697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1736754-31a9-45bf-a217-36c55a37c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.371396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1736754-31a9-45bf-a217-36c55a37c927 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.371702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1736754-31a9-45bf-a217-36c55a37c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.371951 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1736754-31a9-45bf-a217-36c55a37c927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.372684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1736754-31a9-45bf-a217-36c55a37c927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:47.374287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1736754-31a9-45bf-a217-36c55a37c927 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:47.386147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1234/2454] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:47.390532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6d7a0-5e6f-41b9-8467-7043d4379ecc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.400979 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6d7a0-5e6f-41b9-8467-7043d4379ecc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.405629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26291537-6bb7-489d-93c1-e8a110fc5144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.406284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26291537-6bb7-489d-93c1-e8a110fc5144 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.406612 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26291537-6bb7-489d-93c1-e8a110fc5144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.406899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26291537-6bb7-489d-93c1-e8a110fc5144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.407631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26291537-6bb7-489d-93c1-e8a110fc5144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:47.409239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26291537-6bb7-489d-93c1-e8a110fc5144 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:47.419483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1221/2455] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:35:47.424857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1736754-31a9-45bf-a217-36c55a37c927 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.434723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1736754-31a9-45bf-a217-36c55a37c927 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.439442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5ddc7-62c8-4f3c-9150-0502377759d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.440342 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5ddc7-62c8-4f3c-9150-0502377759d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.440620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5ddc7-62c8-4f3c-9150-0502377759d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.440917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5ddc7-62c8-4f3c-9150-0502377759d7 admin admin] PATH_INFO: `/v3/projects/137cf000b4c5464586459c0607882ad5/users/561cbeb866e6458aa510e9bf10116055/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.456424 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5ddc7-62c8-4f3c-9150-0502377759d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=137cf000b4c5464586459c0607882ad5, user_id=561cbeb866e6458aa510e9bf10116055, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:47.459264 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5ddc7-62c8-4f3c-9150-0502377759d7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:47.530473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1235/2456] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:35:47 2022] PUT /identity/v3/projects/137cf000b4c5464586459c0607882ad5/users/561cbeb866e6458aa510e9bf10116055/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:47.541040 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167e89-4f15-4564-9d48-b5c80043e153 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.541360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167e89-4f15-4564-9d48-b5c80043e153 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.541640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167e89-4f15-4564-9d48-b5c80043e153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.571973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cb167e89-4f15-4564-9d48-b5c80043e153 None None] MFA Rules not processed for user `561cbeb866e6458aa510e9bf10116055`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:47.573268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cb167e89-4f15-4564-9d48-b5c80043e153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:47.649652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1222/2457] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:47.654669 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5ddc7-62c8-4f3c-9150-0502377759d7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.722689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5ddc7-62c8-4f3c-9150-0502377759d7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.728558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dad2b-78ea-4956-af9b-96112aada0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.729214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dad2b-78ea-4956-af9b-96112aada0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.729495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dad2b-78ea-4956-af9b-96112aada0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.729770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dad2b-78ea-4956-af9b-96112aada0a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.730508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dad2b-78ea-4956-af9b-96112aada0a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:47.732414 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dad2b-78ea-4956-af9b-96112aada0a0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:47.795773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1236/2458] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:35:47 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:47.800330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb167e89-4f15-4564-9d48-b5c80043e153 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.812272 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb167e89-4f15-4564-9d48-b5c80043e153 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.852659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:47.869284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c7ad7-b4de-45e7-8ed2-237e843d5ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.869978 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c7ad7-b4de-45e7-8ed2-237e843d5ecb admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.870303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c7ad7-b4de-45e7-8ed2-237e843d5ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.870617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c7ad7-b4de-45e7-8ed2-237e843d5ecb admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.871314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c7ad7-b4de-45e7-8ed2-237e843d5ecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:47.873279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c7ad7-b4de-45e7-8ed2-237e843d5ecb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:47.924663 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1223/2459] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:35:47 2022] POST /identity/v3/users => generated 300 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:47.929310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dad2b-78ea-4956-af9b-96112aada0a0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.940116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dad2b-78ea-4956-af9b-96112aada0a0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.945229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61ee47-63a5-4241-aba1-ebd89dd27ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.945892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61ee47-63a5-4241-aba1-ebd89dd27ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.946182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61ee47-63a5-4241-aba1-ebd89dd27ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.946503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61ee47-63a5-4241-aba1-ebd89dd27ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:47.947233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61ee47-63a5-4241-aba1-ebd89dd27ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:47.948844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61ee47-63a5-4241-aba1-ebd89dd27ca4 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:47.963871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1237/2460] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:47.968087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c7ad7-b4de-45e7-8ed2-237e843d5ecb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:47.978837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c7ad7-b4de-45e7-8ed2-237e843d5ecb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:47.983615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca9ff5-8bcc-4fad-a103-fae4696aeed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:47.984499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca9ff5-8bcc-4fad-a103-fae4696aeed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:47.984784 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca9ff5-8bcc-4fad-a103-fae4696aeed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:47.985060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca9ff5-8bcc-4fad-a103-fae4696aeed4 admin admin] PATH_INFO: `/v3/projects/23b201e25bed46fbbc34cbb060711fea/users/9ca088c7582540d8b2f4dafcace46a71/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:48.000291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca9ff5-8bcc-4fad-a103-fae4696aeed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b201e25bed46fbbc34cbb060711fea, user_id=9ca088c7582540d8b2f4dafcace46a71, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:48.003011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca9ff5-8bcc-4fad-a103-fae4696aeed4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:48.119668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1224/2461] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:35:47 2022] PUT /identity/v3/projects/23b201e25bed46fbbc34cbb060711fea/users/9ca088c7582540d8b2f4dafcace46a71/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:48.123792 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61ee47-63a5-4241-aba1-ebd89dd27ca4 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:48.133921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61ee47-63a5-4241-aba1-ebd89dd27ca4 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:48.154576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:48.191937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd42cb4-8e9a-4dda-a2c7-a3bbd99ee218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:48.192569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd42cb4-8e9a-4dda-a2c7-a3bbd99ee218 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:48.192857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd42cb4-8e9a-4dda-a2c7-a3bbd99ee218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:48.193139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd42cb4-8e9a-4dda-a2c7-a3bbd99ee218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:48.193872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd42cb4-8e9a-4dda-a2c7-a3bbd99ee218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:48.195698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd42cb4-8e9a-4dda-a2c7-a3bbd99ee218 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:48.212431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1238/2462] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:48.212597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca9ff5-8bcc-4fad-a103-fae4696aeed4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:48.221075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca9ff5-8bcc-4fad-a103-fae4696aeed4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:48.226330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c967eb-0faa-463f-95f0-611fd3ed9118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:48.227244 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c967eb-0faa-463f-95f0-611fd3ed9118 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:48.227526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c967eb-0faa-463f-95f0-611fd3ed9118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:48.227801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c967eb-0faa-463f-95f0-611fd3ed9118 admin admin] PATH_INFO: `/v3/projects/23b201e25bed46fbbc34cbb060711fea/users/9ca088c7582540d8b2f4dafcace46a71/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:48.229749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c967eb-0faa-463f-95f0-611fd3ed9118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b201e25bed46fbbc34cbb060711fea, user_id=9ca088c7582540d8b2f4dafcace46a71, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:48.232555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c967eb-0faa-463f-95f0-611fd3ed9118 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:48.243327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1225/2463] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:35:48 2022] PUT /identity/v3/projects/23b201e25bed46fbbc34cbb060711fea/users/9ca088c7582540d8b2f4dafcace46a71/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:48.259018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148f6b8-a6ac-4d11-bd10-0b3e62fc02a3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:48.259317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148f6b8-a6ac-4d11-bd10-0b3e62fc02a3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:48.259600 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148f6b8-a6ac-4d11-bd10-0b3e62fc02a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:48.290217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-3148f6b8-a6ac-4d11-bd10-0b3e62fc02a3 None None] MFA Rules not processed for user `9ca088c7582540d8b2f4dafcace46a71`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:48.291810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3148f6b8-a6ac-4d11-bd10-0b3e62fc02a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:48.371982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1239/2464] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:48.384625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c967eb-0faa-463f-95f0-611fd3ed9118 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:48.391900 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-08c967eb-0faa-463f-95f0-611fd3ed9118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:48.443788 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c967eb-0faa-463f-95f0-611fd3ed9118 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:48.447944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e87a46-3a9b-4d6f-88fc-5a232e4800ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:48.448584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e87a46-3a9b-4d6f-88fc-5a232e4800ed service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:48.448866 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e87a46-3a9b-4d6f-88fc-5a232e4800ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:48.449144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e87a46-3a9b-4d6f-88fc-5a232e4800ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:48.458492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e87a46-3a9b-4d6f-88fc-5a232e4800ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:48.460764 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e87a46-3a9b-4d6f-88fc-5a232e4800ed service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:48.470238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1226/2465] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:35:48.533806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148f6b8-a6ac-4d11-bd10-0b3e62fc02a3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:48.596588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148f6b8-a6ac-4d11-bd10-0b3e62fc02a3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:48.601339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:48.602207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:48.602540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:48.602822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] PATH_INFO: `/v3/users/561cbeb866e6458aa510e9bf10116055` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:48.604009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=561cbeb866e6458aa510e9bf10116055)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:48.605917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:48.784993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561cbeb866e6458aa510e9bf10116055'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:48.786171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:48.792843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561cbeb866e6458aa510e9bf10116055'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:48.793528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561cbeb866e6458aa510e9bf10116055'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:48.794144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561cbeb866e6458aa510e9bf10116055'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:48.838601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561cbeb866e6458aa510e9bf10116055'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:48.856412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1240/2466] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:35:48 2022] DELETE /identity/v3/users/561cbeb866e6458aa510e9bf10116055 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:48.861102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3ff31-8e16-40c8-9ff1-cd6fea35e31b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:48.861391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3ff31-8e16-40c8-9ff1-cd6fea35e31b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:48.861671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3ff31-8e16-40c8-9ff1-cd6fea35e31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:48.879948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e6d3ff31-8e16-40c8-9ff1-cd6fea35e31b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:48.881128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e6d3ff31-8e16-40c8-9ff1-cd6fea35e31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:48.941066 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1227/2467] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:48.956881 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:48.958416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:49.014764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351298c-9368-451c-81c1-5a8b35cefd1c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:49.018419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b352eb-be91-439f-aa76-de9da7e1aa02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:49.019078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b352eb-be91-439f-aa76-de9da7e1aa02 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:49.019359 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b352eb-be91-439f-aa76-de9da7e1aa02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:49.019632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b352eb-be91-439f-aa76-de9da7e1aa02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:49.032439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b352eb-be91-439f-aa76-de9da7e1aa02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:49.033900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b352eb-be91-439f-aa76-de9da7e1aa02 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:49.044279 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1241/2468] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:35:49.499560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3ff31-8e16-40c8-9ff1-cd6fea35e31b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:49.501054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e6d3ff31-8e16-40c8-9ff1-cd6fea35e31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:49.513359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3ff31-8e16-40c8-9ff1-cd6fea35e31b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:49.518367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:49.519275 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:49.519557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:49.519831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] PATH_INFO: `/v3/projects/137cf000b4c5464586459c0607882ad5` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:49.521035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] RBAC: Authorizing `identity:delete_project(project_id=137cf000b4c5464586459c0607882ad5)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:49.524115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:49.656162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:49.668704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '137cf000b4c5464586459c0607882ad5', 'request_id': 'req-aee0d91a-5ef6-4a38-8705-44125c81d29c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:49.674803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '137cf000b4c5464586459c0607882ad5', 'request_id': 'req-aee0d91a-5ef6-4a38-8705-44125c81d29c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:49.822600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '137cf000b4c5464586459c0607882ad5', 'request_id': 'req-aee0d91a-5ef6-4a38-8705-44125c81d29c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:49.835092 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] The token cache is being invalidate because project 137cf000b4c5464586459c0607882ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:35:49.836037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:49.836557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:49.838123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1228/2469] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:35:49 2022] DELETE /identity/v3/projects/137cf000b4c5464586459c0607882ad5 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:49.842129 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b352eb-be91-439f-aa76-de9da7e1aa02 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:49.852694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b352eb-be91-439f-aa76-de9da7e1aa02 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:49.908831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:49.909632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:49.909914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:49.910188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] PATH_INFO: `/v3/users/9ca088c7582540d8b2f4dafcace46a71` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:49.911403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca088c7582540d8b2f4dafcace46a71)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:49.913501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:50.236154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca088c7582540d8b2f4dafcace46a71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:50.243548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca088c7582540d8b2f4dafcace46a71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:50.244207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca088c7582540d8b2f4dafcace46a71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:50.244809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca088c7582540d8b2f4dafcace46a71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:50.353022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f5154754-c88d-4b4f-ac87-39ca8a53200f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca088c7582540d8b2f4dafcace46a71'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:50.358973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:50.371762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1242/2470] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:35:49 2022] DELETE /identity/v3/users/9ca088c7582540d8b2f4dafcace46a71 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:50.854205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:50.855933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:50.870660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0d91a-5ef6-4a38-8705-44125c81d29c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:50.934103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:50.935194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:50.935481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:50.935761 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] PATH_INFO: `/v3/projects/23b201e25bed46fbbc34cbb060711fea` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:50.937166 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b201e25bed46fbbc34cbb060711fea)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:50.939184 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:51.059358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b201e25bed46fbbc34cbb060711fea', 'request_id': 'req-779b8798-685f-4fa2-abc9-2badcac96bb4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:51.065686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b201e25bed46fbbc34cbb060711fea', 'request_id': 'req-779b8798-685f-4fa2-abc9-2badcac96bb4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:51.137064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b201e25bed46fbbc34cbb060711fea', 'request_id': 'req-779b8798-685f-4fa2-abc9-2badcac96bb4'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:51.150357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] The token cache is being invalidate because project 23b201e25bed46fbbc34cbb060711fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:35:51.150681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:51.151321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-779b8798-685f-4fa2-abc9-2badcac96bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:51.153109 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1229/2471] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:35:50 2022] DELETE /identity/v3/projects/23b201e25bed46fbbc34cbb060711fea => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:51.159102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac959a-3f34-4dac-863c-cb163282363d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:51.159573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac959a-3f34-4dac-863c-cb163282363d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:51.159668 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac959a-3f34-4dac-863c-cb163282363d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:51.179380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-92ac959a-3f34-4dac-863c-cb163282363d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:51.180901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-92ac959a-3f34-4dac-863c-cb163282363d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:51.243458 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1243/2472] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:51.251517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cc2c1-7ef3-482b-acf1-6f9fc8150aaa None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:51.251811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cc2c1-7ef3-482b-acf1-6f9fc8150aaa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:51.252089 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cc2c1-7ef3-482b-acf1-6f9fc8150aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:51.271474 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1a7cc2c1-7ef3-482b-acf1-6f9fc8150aaa None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:51.272950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1a7cc2c1-7ef3-482b-acf1-6f9fc8150aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:51.281010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1230/2473] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:51.285354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac959a-3f34-4dac-863c-cb163282363d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:51.295200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac959a-3f34-4dac-863c-cb163282363d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:51.299992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cf520-29f5-415a-980d-28f451ffb904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:51.300644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cf520-29f5-415a-980d-28f451ffb904 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:51.300927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cf520-29f5-415a-980d-28f451ffb904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:51.301201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cf520-29f5-415a-980d-28f451ffb904 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:51.301924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cf520-29f5-415a-980d-28f451ffb904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:35:51.302236 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cf520-29f5-415a-980d-28f451ffb904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:51.304378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cf520-29f5-415a-980d-28f451ffb904 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:51.316604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1244/2474] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:35:51.321622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cc2c1-7ef3-482b-acf1-6f9fc8150aaa None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:51.331084 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cc2c1-7ef3-482b-acf1-6f9fc8150aaa None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:51.335926 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e1b72-b77b-4556-a57e-cc7413029101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:51.336583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e1b72-b77b-4556-a57e-cc7413029101 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:51.336868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e1b72-b77b-4556-a57e-cc7413029101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:51.337149 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e1b72-b77b-4556-a57e-cc7413029101 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:51.337850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e1b72-b77b-4556-a57e-cc7413029101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:51.339798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e1b72-b77b-4556-a57e-cc7413029101 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:51.435728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1231/2475] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:35:51 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:51.439932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cf520-29f5-415a-980d-28f451ffb904 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:51.450566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cf520-29f5-415a-980d-28f451ffb904 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:51.508575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31ce6f-103e-4d6c-b37b-4288e6e853ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:51.509244 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31ce6f-103e-4d6c-b37b-4288e6e853ed admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:51.509526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31ce6f-103e-4d6c-b37b-4288e6e853ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:51.509800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31ce6f-103e-4d6c-b37b-4288e6e853ed admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:51.510524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31ce6f-103e-4d6c-b37b-4288e6e853ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:51.512441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31ce6f-103e-4d6c-b37b-4288e6e853ed admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:51.607530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1245/2476] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:35:51 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:51.613887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e1b72-b77b-4556-a57e-cc7413029101 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:51.625087 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e1b72-b77b-4556-a57e-cc7413029101 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:51.630176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6e32-ce04-4e36-be0d-8fb1c9471f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:51.630886 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6e32-ce04-4e36-be0d-8fb1c9471f86 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:51.631195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6e32-ce04-4e36-be0d-8fb1c9471f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:51.632034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f6e32-ce04-4e36-be0d-8fb1c9471f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:51.632793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6e32-ce04-4e36-be0d-8fb1c9471f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:51.634395 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f6e32-ce04-4e36-be0d-8fb1c9471f86 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:51.645139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1232/2477] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:51.649741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31ce6f-103e-4d6c-b37b-4288e6e853ed admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:51.659777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31ce6f-103e-4d6c-b37b-4288e6e853ed admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:51.664906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800944a3-cf40-41bf-a170-a38d7cc315a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:51.665539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800944a3-cf40-41bf-a170-a38d7cc315a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:51.665821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800944a3-cf40-41bf-a170-a38d7cc315a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:51.666096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800944a3-cf40-41bf-a170-a38d7cc315a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:51.666852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800944a3-cf40-41bf-a170-a38d7cc315a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:51.668462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800944a3-cf40-41bf-a170-a38d7cc315a6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:51.678461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1246/2478] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:51.694451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6e32-ce04-4e36-be0d-8fb1c9471f86 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:51.705990 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f6e32-ce04-4e36-be0d-8fb1c9471f86 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:51.712618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0ba29-1c94-4249-8d35-771f1c522352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:51.713517 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0ba29-1c94-4249-8d35-771f1c522352 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:51.713931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0ba29-1c94-4249-8d35-771f1c522352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:51.714215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0ba29-1c94-4249-8d35-771f1c522352 admin admin] PATH_INFO: `/v3/projects/cf6af8a095ac48f7800e94f39d26f597/users/0d6ff10d60f74170b94ac755cebceb51/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:51.722697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:51.731120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0ba29-1c94-4249-8d35-771f1c522352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6af8a095ac48f7800e94f39d26f597, user_id=0d6ff10d60f74170b94ac755cebceb51, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:51.734002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac0ba29-1c94-4249-8d35-771f1c522352 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:51.839624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1233/2479] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:35:51 2022] PUT /identity/v3/projects/cf6af8a095ac48f7800e94f39d26f597/users/0d6ff10d60f74170b94ac755cebceb51/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:51.849809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bac29-0306-4654-9d4c-2caf6e7ab470 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:51.850313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bac29-0306-4654-9d4c-2caf6e7ab470 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:51.850511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bac29-0306-4654-9d4c-2caf6e7ab470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:51.877871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-912bac29-0306-4654-9d4c-2caf6e7ab470 None None] MFA Rules not processed for user `0d6ff10d60f74170b94ac755cebceb51`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:51.879215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-912bac29-0306-4654-9d4c-2caf6e7ab470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:51.926243 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:51.949415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1247/2480] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:51.970297 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0ba29-1c94-4249-8d35-771f1c522352 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:51.972059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8ac0ba29-1c94-4249-8d35-771f1c522352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:52.030140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0ba29-1c94-4249-8d35-771f1c522352 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:52.033792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956f8c9-0c44-44d9-8be0-595a5209d521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:52.034460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956f8c9-0c44-44d9-8be0-595a5209d521 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:52.034748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956f8c9-0c44-44d9-8be0-595a5209d521 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:52.035030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956f8c9-0c44-44d9-8be0-595a5209d521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:52.045198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956f8c9-0c44-44d9-8be0-595a5209d521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:52.051780 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956f8c9-0c44-44d9-8be0-595a5209d521 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:52.057317 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1234/2481] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:35:52.090804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bac29-0306-4654-9d4c-2caf6e7ab470 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:52.154869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bac29-0306-4654-9d4c-2caf6e7ab470 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:52.159505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:52.160381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:52.160667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:52.160949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] PATH_INFO: `/v3/users/0d6ff10d60f74170b94ac755cebceb51` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:52.162153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6ff10d60f74170b94ac755cebceb51)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:52.164412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:52.362691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6ff10d60f74170b94ac755cebceb51'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:52.370546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6ff10d60f74170b94ac755cebceb51'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:52.371307 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6ff10d60f74170b94ac755cebceb51'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:52.371982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6ff10d60f74170b94ac755cebceb51'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:52.507692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6ff10d60f74170b94ac755cebceb51'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:52.528433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1248/2482] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:35:52 2022] DELETE /identity/v3/users/0d6ff10d60f74170b94ac755cebceb51 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:35:52.532962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1d6de-829e-43e1-84f0-fc22fdd38b3c None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:52.533283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1d6de-829e-43e1-84f0-fc22fdd38b3c None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:52.533572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1d6de-829e-43e1-84f0-fc22fdd38b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:52.552161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-7ff1d6de-829e-43e1-84f0-fc22fdd38b3c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:52.553824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7ff1d6de-829e-43e1-84f0-fc22fdd38b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:52.613448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1235/2483] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:52.623271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:52.624689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:52.682542 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be4f4-4204-4f91-bdce-649cada05cd8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:52.686479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61305ecb-d2be-4136-b869-41a3e7740670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:52.687151 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61305ecb-d2be-4136-b869-41a3e7740670 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:52.687433 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61305ecb-d2be-4136-b869-41a3e7740670 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:52.687708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61305ecb-d2be-4136-b869-41a3e7740670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:52.697406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61305ecb-d2be-4136-b869-41a3e7740670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:52.698916 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61305ecb-d2be-4136-b869-41a3e7740670 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:52.710106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1249/2484] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:35:53.103296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1d6de-829e-43e1-84f0-fc22fdd38b3c None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:53.104912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7ff1d6de-829e-43e1-84f0-fc22fdd38b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:53.120624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1d6de-829e-43e1-84f0-fc22fdd38b3c None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:53.128379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:53.129303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:53.130911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:53.131781 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] PATH_INFO: `/v3/projects/cf6af8a095ac48f7800e94f39d26f597` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:53.133633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6af8a095ac48f7800e94f39d26f597)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:53.135709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:53.241571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6af8a095ac48f7800e94f39d26f597', 'request_id': 'req-4639ebfd-f429-4f3a-a700-2392108fb71c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:53.247651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6af8a095ac48f7800e94f39d26f597', 'request_id': 'req-4639ebfd-f429-4f3a-a700-2392108fb71c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:53.307649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6af8a095ac48f7800e94f39d26f597', 'request_id': 'req-4639ebfd-f429-4f3a-a700-2392108fb71c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:53.320010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] The token cache is being invalidate because project cf6af8a095ac48f7800e94f39d26f597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:35:53.320291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:53.320927 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-4639ebfd-f429-4f3a-a700-2392108fb71c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:35:53.322479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1236/2485] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:35:53 2022] DELETE /identity/v3/projects/cf6af8a095ac48f7800e94f39d26f597 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:35:53.327550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380f0cb-d7cf-469d-b5e1-dfe6792f48fa None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:53.328039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380f0cb-d7cf-469d-b5e1-dfe6792f48fa None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:53.328170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380f0cb-d7cf-469d-b5e1-dfe6792f48fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:53.335613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:53.347679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-a380f0cb-d7cf-469d-b5e1-dfe6792f48fa None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:53.348891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a380f0cb-d7cf-469d-b5e1-dfe6792f48fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:53.407807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1250/2486] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:53.416774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89febfad-2d1e-4d20-a8b6-6743f38264db None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:53.417060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89febfad-2d1e-4d20-a8b6-6743f38264db None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:53.419253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89febfad-2d1e-4d20-a8b6-6743f38264db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:53.436837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-89febfad-2d1e-4d20-a8b6-6743f38264db None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:53.438070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-89febfad-2d1e-4d20-a8b6-6743f38264db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:53.446857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1237/2487] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:53.451484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380f0cb-d7cf-469d-b5e1-dfe6792f48fa None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:53.461362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380f0cb-d7cf-469d-b5e1-dfe6792f48fa None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:53.467143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de12dc-f734-4fad-b4ca-1c881d89a635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:53.467828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de12dc-f734-4fad-b4ca-1c881d89a635 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:53.468130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de12dc-f734-4fad-b4ca-1c881d89a635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:53.468397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de12dc-f734-4fad-b4ca-1c881d89a635 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:53.469155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de12dc-f734-4fad-b4ca-1c881d89a635 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:35:53.469482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de12dc-f734-4fad-b4ca-1c881d89a635 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:53.471184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de12dc-f734-4fad-b4ca-1c881d89a635 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:53.482992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1251/2488] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:35:53.487431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89febfad-2d1e-4d20-a8b6-6743f38264db None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:53.497652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89febfad-2d1e-4d20-a8b6-6743f38264db None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:53.502369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c63c8e-a7e6-41df-92c3-fec41daaefa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:53.503033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c63c8e-a7e6-41df-92c3-fec41daaefa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:53.503319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c63c8e-a7e6-41df-92c3-fec41daaefa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:53.503604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c63c8e-a7e6-41df-92c3-fec41daaefa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:53.504302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c63c8e-a7e6-41df-92c3-fec41daaefa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:53.506224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c63c8e-a7e6-41df-92c3-fec41daaefa1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:53.510002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:53.633933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1238/2489] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:35:53 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:35:53.635401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de12dc-f734-4fad-b4ca-1c881d89a635 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:53.645498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de12dc-f734-4fad-b4ca-1c881d89a635 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:53.704625 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11197cd7-38ed-4586-afd9-5ec9d0124aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:53.705208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11197cd7-38ed-4586-afd9-5ec9d0124aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:53.705492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11197cd7-38ed-4586-afd9-5ec9d0124aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:53.705820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11197cd7-38ed-4586-afd9-5ec9d0124aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:53.706567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11197cd7-38ed-4586-afd9-5ec9d0124aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:53.708473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11197cd7-38ed-4586-afd9-5ec9d0124aa9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:53.816584 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1252/2490] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:35:53 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:35:53.821409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c63c8e-a7e6-41df-92c3-fec41daaefa1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:53.831551 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c63c8e-a7e6-41df-92c3-fec41daaefa1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:53.836355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62c954-cbc6-4a02-ab68-2951e9bc3528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:53.837008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62c954-cbc6-4a02-ab68-2951e9bc3528 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:53.837330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62c954-cbc6-4a02-ab68-2951e9bc3528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:53.837607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62c954-cbc6-4a02-ab68-2951e9bc3528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:53.838326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62c954-cbc6-4a02-ab68-2951e9bc3528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:53.839948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62c954-cbc6-4a02-ab68-2951e9bc3528 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:53.850525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1239/2491] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:53.854515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11197cd7-38ed-4586-afd9-5ec9d0124aa9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:53.864415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11197cd7-38ed-4586-afd9-5ec9d0124aa9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:53.870088 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b739fab-abc6-452b-85b0-9b1b75e45e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:53.870785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739fab-abc6-452b-85b0-9b1b75e45e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:53.871366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739fab-abc6-452b-85b0-9b1b75e45e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:53.871656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739fab-abc6-452b-85b0-9b1b75e45e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:53.872497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739fab-abc6-452b-85b0-9b1b75e45e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:53.874105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739fab-abc6-452b-85b0-9b1b75e45e20 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:53.884999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1253/2492] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:35:53.889118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62c954-cbc6-4a02-ab68-2951e9bc3528 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:53.899276 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62c954-cbc6-4a02-ab68-2951e9bc3528 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:53.904063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabb93d-a93d-495c-8c9e-f6c18017b6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:53.904948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabb93d-a93d-495c-8c9e-f6c18017b6b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:53.905240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabb93d-a93d-495c-8c9e-f6c18017b6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:53.905524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabb93d-a93d-495c-8c9e-f6c18017b6b2 admin admin] PATH_INFO: `/v3/projects/0558cab0498a4ccb830fe7ad72066401/users/05cef7fc49ad4901851d9888cbbc91fa/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:53.920503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabb93d-a93d-495c-8c9e-f6c18017b6b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0558cab0498a4ccb830fe7ad72066401, user_id=05cef7fc49ad4901851d9888cbbc91fa, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:53.923228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabb93d-a93d-495c-8c9e-f6c18017b6b2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:53.993658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1240/2493] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:35:53 2022] PUT /identity/v3/projects/0558cab0498a4ccb830fe7ad72066401/users/05cef7fc49ad4901851d9888cbbc91fa/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:35:54.003721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccd931-f872-471b-883e-99dade9632ac None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:54.004013 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccd931-f872-471b-883e-99dade9632ac None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:54.004301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccd931-f872-471b-883e-99dade9632ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:54.032965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b9ccd931-f872-471b-883e-99dade9632ac None None] MFA Rules not processed for user `05cef7fc49ad4901851d9888cbbc91fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:35:54.034170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b9ccd931-f872-471b-883e-99dade9632ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:54.096931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:35:54.100575 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1254/2494] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:35:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:35:54.112728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabb93d-a93d-495c-8c9e-f6c18017b6b2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:54.114439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6eabb93d-a93d-495c-8c9e-f6c18017b6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:54.172110 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabb93d-a93d-495c-8c9e-f6c18017b6b2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:54.176075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b540ab-3051-4488-84e3-a1b6e9a076f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9969632e796345fa9671a45cf61ede3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:54.181958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b540ab-3051-4488-84e3-a1b6e9a076f5 service glance] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:54.181958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b540ab-3051-4488-84e3-a1b6e9a076f5 service glance] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:54.181958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b540ab-3051-4488-84e3-a1b6e9a076f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:54.187337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b540ab-3051-4488-84e3-a1b6e9a076f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:54.188810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b540ab-3051-4488-84e3-a1b6e9a076f5 service glance] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:54.198302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1241/2495] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:35:54.546794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccd931-f872-471b-883e-99dade9632ac None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:35:54.548247 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b9ccd931-f872-471b-883e-99dade9632ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:54.608086 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccd931-f872-471b-883e-99dade9632ac None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:35:54.612005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ff78e-ee2c-49db-b2f5-ee2b4835ff11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:35:54.612652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ff78e-ee2c-49db-b2f5-ee2b4835ff11 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:35:54.612933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ff78e-ee2c-49db-b2f5-ee2b4835ff11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:35:54.613207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ff78e-ee2c-49db-b2f5-ee2b4835ff11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:35:54.615301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-959ff78e-ee2c-49db-b2f5-ee2b4835ff11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:35:54.699984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ff78e-ee2c-49db-b2f5-ee2b4835ff11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:35:54.701437 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ff78e-ee2c-49db-b2f5-ee2b4835ff11 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:35:54.723578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1255/2496] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:35:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:01.456750 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b540ab-3051-4488-84e3-a1b6e9a076f5 service glance] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:01.458677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-11b540ab-3051-4488-84e3-a1b6e9a076f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:01.547248 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b540ab-3051-4488-84e3-a1b6e9a076f5 service glance] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:01.551914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:01.553625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:01.553625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:01.553625 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] PATH_INFO: `/v3/users/8f609cfe0d7a4ded8a622c65d8690f7f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:01.554661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f609cfe0d7a4ded8a622c65d8690f7f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:01.556867 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:01.816115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f609cfe0d7a4ded8a622c65d8690f7f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:01.816793 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f609cfe0d7a4ded8a622c65d8690f7f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:01.823650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:01.831528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f609cfe0d7a4ded8a622c65d8690f7f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:01.832301 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f609cfe0d7a4ded8a622c65d8690f7f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:01.931011 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-882f52fd-f17e-4366-bd6c-fcbe264847fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f609cfe0d7a4ded8a622c65d8690f7f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:01.944064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1242/2497] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:01 2022] DELETE /identity/v3/users/8f609cfe0d7a4ded8a622c65d8690f7f => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:02.171945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ff78e-ee2c-49db-b2f5-ee2b4835ff11 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:02.184227 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ff78e-ee2c-49db-b2f5-ee2b4835ff11 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:02.242157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:02.243079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:02.243369 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:02.243671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] PATH_INFO: `/v3/projects/886ee445e38e4c31bd4d4d014a7c4c7c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:02.244857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=886ee445e38e4c31bd4d4d014a7c4c7c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:02.247759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:02.321564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:02.337153 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '886ee445e38e4c31bd4d4d014a7c4c7c', 'request_id': 'req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.343218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '886ee445e38e4c31bd4d4d014a7c4c7c', 'request_id': 'req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.416095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '886ee445e38e4c31bd4d4d014a7c4c7c', 'request_id': 'req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.429017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] The token cache is being invalidate because project 886ee445e38e4c31bd4d4d014a7c4c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:02.429311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.429956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ee5489c3-d977-49b3-b8aa-2a93c58dbd39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:02.431790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1256/2498] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:02 2022] DELETE /identity/v3/projects/886ee445e38e4c31bd4d4d014a7c4c7c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:02.442000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579d5df-0513-4193-b5f4-200a489d7fcf None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:02.442330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579d5df-0513-4193-b5f4-200a489d7fcf None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:02.442721 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579d5df-0513-4193-b5f4-200a489d7fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:02.461624 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6579d5df-0513-4193-b5f4-200a489d7fcf None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:02.462871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6579d5df-0513-4193-b5f4-200a489d7fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:02.531331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1243/2499] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:02.540031 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818a13e-dc9c-4ea2-ba80-57a047036fac None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:02.540382 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818a13e-dc9c-4ea2-ba80-57a047036fac None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:02.540679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818a13e-dc9c-4ea2-ba80-57a047036fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:02.563460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f818a13e-dc9c-4ea2-ba80-57a047036fac None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:02.564665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f818a13e-dc9c-4ea2-ba80-57a047036fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:02.575341 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1257/2500] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:02.578505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579d5df-0513-4193-b5f4-200a489d7fcf None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:02.590032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579d5df-0513-4193-b5f4-200a489d7fcf None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:02.593853 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e9472-8533-4fc4-8e9a-5c355c77e966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:02.594548 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e9472-8533-4fc4-8e9a-5c355c77e966 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:02.594830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e9472-8533-4fc4-8e9a-5c355c77e966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:02.595238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e9472-8533-4fc4-8e9a-5c355c77e966 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:02.596006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e9472-8533-4fc4-8e9a-5c355c77e966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:36:02.596322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e9472-8533-4fc4-8e9a-5c355c77e966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:02.597958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e9472-8533-4fc4-8e9a-5c355c77e966 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:02.611068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1244/2501] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:36:02.615549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818a13e-dc9c-4ea2-ba80-57a047036fac None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:02.625577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818a13e-dc9c-4ea2-ba80-57a047036fac None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:02.630755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3b74e-8a35-4796-85de-a8fd86fbc64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:02.631466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3b74e-8a35-4796-85de-a8fd86fbc64c admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:02.631757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3b74e-8a35-4796-85de-a8fd86fbc64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:02.632783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3b74e-8a35-4796-85de-a8fd86fbc64c admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:02.632893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3b74e-8a35-4796-85de-a8fd86fbc64c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:02.634813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3b74e-8a35-4796-85de-a8fd86fbc64c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:02.732661 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1258/2502] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:02 2022] POST /identity/v3/projects => generated 399 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:02.737095 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e9472-8533-4fc4-8e9a-5c355c77e966 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:02.748008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e9472-8533-4fc4-8e9a-5c355c77e966 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:02.806952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe535c10-d112-4092-896d-4ac1567366a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:02.807629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe535c10-d112-4092-896d-4ac1567366a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:02.807914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe535c10-d112-4092-896d-4ac1567366a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:02.808881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe535c10-d112-4092-896d-4ac1567366a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:02.809611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe535c10-d112-4092-896d-4ac1567366a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:02.811601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe535c10-d112-4092-896d-4ac1567366a6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:02.914008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1245/2503] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:02 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:02.918849 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3b74e-8a35-4796-85de-a8fd86fbc64c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:02.929248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3b74e-8a35-4796-85de-a8fd86fbc64c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:02.934304 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06747c3f-634a-4def-a39b-b7de41059ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:02.934931 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06747c3f-634a-4def-a39b-b7de41059ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:02.940248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06747c3f-634a-4def-a39b-b7de41059ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:02.940248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06747c3f-634a-4def-a39b-b7de41059ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:02.940248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06747c3f-634a-4def-a39b-b7de41059ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:02.940248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06747c3f-634a-4def-a39b-b7de41059ce3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:02.948647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1259/2504] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:02.954080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe535c10-d112-4092-896d-4ac1567366a6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:02.968930 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe535c10-d112-4092-896d-4ac1567366a6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:02.977073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f40d5e-8c27-4a94-9878-1c8727ff92fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:02.978234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f40d5e-8c27-4a94-9878-1c8727ff92fc admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:02.978776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f40d5e-8c27-4a94-9878-1c8727ff92fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:02.979279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f40d5e-8c27-4a94-9878-1c8727ff92fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:02.981344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f40d5e-8c27-4a94-9878-1c8727ff92fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:02.983167 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f40d5e-8c27-4a94-9878-1c8727ff92fc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:02.994032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1246/2505] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:02.998616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06747c3f-634a-4def-a39b-b7de41059ce3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:03.008621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06747c3f-634a-4def-a39b-b7de41059ce3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:03.013393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ab784-5c86-45b3-9fe5-639c5c67690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:03.014262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ab784-5c86-45b3-9fe5-639c5c67690f admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:03.014594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ab784-5c86-45b3-9fe5-639c5c67690f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:03.014943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ab784-5c86-45b3-9fe5-639c5c67690f admin admin] PATH_INFO: `/v3/projects/7ff23bff1a234db08b13c7b93127f279/users/95f4e782ac72403c981a0daa2406973d/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:03.031603 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ab784-5c86-45b3-9fe5-639c5c67690f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff23bff1a234db08b13c7b93127f279, user_id=95f4e782ac72403c981a0daa2406973d, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:03.034446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ab784-5c86-45b3-9fe5-639c5c67690f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:03.114051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1260/2506] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:02 2022] PUT /identity/v3/projects/7ff23bff1a234db08b13c7b93127f279/users/95f4e782ac72403c981a0daa2406973d/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:03.129659 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36ad77-1e95-48c8-a485-8716b1976048 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:03.129950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36ad77-1e95-48c8-a485-8716b1976048 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:03.130241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36ad77-1e95-48c8-a485-8716b1976048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:03.157444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6c36ad77-1e95-48c8-a485-8716b1976048 None None] MFA Rules not processed for user `95f4e782ac72403c981a0daa2406973d`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:03.162797 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6c36ad77-1e95-48c8-a485-8716b1976048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:03.185182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:03.223911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1247/2507] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:03.236525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ab784-5c86-45b3-9fe5-639c5c67690f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:03.238284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-a97ab784-5c86-45b3-9fe5-639c5c67690f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:03.313297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ab784-5c86-45b3-9fe5-639c5c67690f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:03.316980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea51ac-e821-45da-972c-57acf7973015 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:03.317623 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea51ac-e821-45da-972c-57acf7973015 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:03.317908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea51ac-e821-45da-972c-57acf7973015 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:03.318186 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea51ac-e821-45da-972c-57acf7973015 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:03.326929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea51ac-e821-45da-972c-57acf7973015 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:03.328371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea51ac-e821-45da-972c-57acf7973015 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:03.337448 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1261/2508] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:03.958165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36ad77-1e95-48c8-a485-8716b1976048 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:03.960215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6c36ad77-1e95-48c8-a485-8716b1976048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:04.018390 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36ad77-1e95-48c8-a485-8716b1976048 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:04.026380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3227-53f6-4f61-b932-5d3e55e4cfcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:04.026380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3227-53f6-4f61-b932-5d3e55e4cfcd service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:04.026380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3227-53f6-4f61-b932-5d3e55e4cfcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:04.026380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3227-53f6-4f61-b932-5d3e55e4cfcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:04.032471 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8fbc3227-53f6-4f61-b932-5d3e55e4cfcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:04.102053 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3227-53f6-4f61-b932-5d3e55e4cfcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:04.105318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3227-53f6-4f61-b932-5d3e55e4cfcd service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:04.122021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1248/2509] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:08.187872 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea51ac-e821-45da-972c-57acf7973015 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:08.189317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e3ea51ac-e821-45da-972c-57acf7973015 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:08.205229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea51ac-e821-45da-972c-57acf7973015 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:08.209927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:08.210837 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:08.211123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:08.211401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] PATH_INFO: `/v3/users/746a70534fe5419ebe6d5e97a7a041f1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:08.212618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746a70534fe5419ebe6d5e97a7a041f1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:08.215100 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:08.413444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:08.418972 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746a70534fe5419ebe6d5e97a7a041f1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:08.426172 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746a70534fe5419ebe6d5e97a7a041f1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:08.426870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746a70534fe5419ebe6d5e97a7a041f1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:08.427483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746a70534fe5419ebe6d5e97a7a041f1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:08.472901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f37502ce-6652-4c30-9fba-51f23474e081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746a70534fe5419ebe6d5e97a7a041f1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:08.491119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1262/2510] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:08 2022] DELETE /identity/v3/users/746a70534fe5419ebe6d5e97a7a041f1 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:08.714264 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3227-53f6-4f61-b932-5d3e55e4cfcd service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:08.725670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3227-53f6-4f61-b932-5d3e55e4cfcd service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:08.787515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:08.788379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:08.788689 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:08.788974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] PATH_INFO: `/v3/projects/1231585c94524ec199c47bd4f4162578` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:08.790142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1231585c94524ec199c47bd4f4162578)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:08.792111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:08.819192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:08.944543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1231585c94524ec199c47bd4f4162578', 'request_id': 'req-a771fb58-7f71-47db-9277-2c661449ca6d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:08.950954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1231585c94524ec199c47bd4f4162578', 'request_id': 'req-a771fb58-7f71-47db-9277-2c661449ca6d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:09.027805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1231585c94524ec199c47bd4f4162578', 'request_id': 'req-a771fb58-7f71-47db-9277-2c661449ca6d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:09.039483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] The token cache is being invalidate because project 1231585c94524ec199c47bd4f4162578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:09.039770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:09.040410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-a771fb58-7f71-47db-9277-2c661449ca6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:09.041936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1249/2511] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:08 2022] DELETE /identity/v3/projects/1231585c94524ec199c47bd4f4162578 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:36:09.047687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f7a95-f09e-43e9-ab1f-61b5951584a3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.047984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f7a95-f09e-43e9-ab1f-61b5951584a3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.048292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f7a95-f09e-43e9-ab1f-61b5951584a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.079371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ae6f7a95-f09e-43e9-ab1f-61b5951584a3 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:09.079371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ae6f7a95-f09e-43e9-ab1f-61b5951584a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:09.127899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1263/2512] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:09.135529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802028c1-bb4f-43e1-89a3-2974f1f38647 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.135823 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802028c1-bb4f-43e1-89a3-2974f1f38647 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.136108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802028c1-bb4f-43e1-89a3-2974f1f38647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.155544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-802028c1-bb4f-43e1-89a3-2974f1f38647 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:09.156754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-802028c1-bb4f-43e1-89a3-2974f1f38647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:09.165091 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1250/2513] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:09.169592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f7a95-f09e-43e9-ab1f-61b5951584a3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:09.179391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f7a95-f09e-43e9-ab1f-61b5951584a3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:09.185199 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e480b10-d8e4-487a-87de-07b140f6e385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:09.185903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e480b10-d8e4-487a-87de-07b140f6e385 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.186190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e480b10-d8e4-487a-87de-07b140f6e385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.186497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e480b10-d8e4-487a-87de-07b140f6e385 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.187231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e480b10-d8e4-487a-87de-07b140f6e385 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:36:09.187561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e480b10-d8e4-487a-87de-07b140f6e385 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:09.189142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e480b10-d8e4-487a-87de-07b140f6e385 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:09.200802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1264/2514] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:36:09.205107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802028c1-bb4f-43e1-89a3-2974f1f38647 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:09.215097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802028c1-bb4f-43e1-89a3-2974f1f38647 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:09.219911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691d206-96d9-4aff-a1d1-8307cc149d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:09.220562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691d206-96d9-4aff-a1d1-8307cc149d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.220850 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691d206-96d9-4aff-a1d1-8307cc149d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.221130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691d206-96d9-4aff-a1d1-8307cc149d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.221831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691d206-96d9-4aff-a1d1-8307cc149d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:09.223770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691d206-96d9-4aff-a1d1-8307cc149d27 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:09.297293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1251/2515] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:09 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:09.301332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e480b10-d8e4-487a-87de-07b140f6e385 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:09.312116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e480b10-d8e4-487a-87de-07b140f6e385 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:09.367896 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861648f-6f0e-4b79-bacc-27f692cd2eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:09.368562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861648f-6f0e-4b79-bacc-27f692cd2eee admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.368847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861648f-6f0e-4b79-bacc-27f692cd2eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.369127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861648f-6f0e-4b79-bacc-27f692cd2eee admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.369818 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861648f-6f0e-4b79-bacc-27f692cd2eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:09.371809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861648f-6f0e-4b79-bacc-27f692cd2eee admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:09.432725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1265/2516] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:09 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:09.437284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691d206-96d9-4aff-a1d1-8307cc149d27 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:09.448114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691d206-96d9-4aff-a1d1-8307cc149d27 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:09.454347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a95f1-d456-4da9-96be-48e37004e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:09.455046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a95f1-d456-4da9-96be-48e37004e535 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.455329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a95f1-d456-4da9-96be-48e37004e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.455607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a95f1-d456-4da9-96be-48e37004e535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.456346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a95f1-d456-4da9-96be-48e37004e535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:09.457970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a95f1-d456-4da9-96be-48e37004e535 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:09.470230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1252/2517] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:09.474269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861648f-6f0e-4b79-bacc-27f692cd2eee admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:09.484801 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861648f-6f0e-4b79-bacc-27f692cd2eee admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:09.489594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443dae85-a03c-4180-a2b3-e857df1401e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:09.490320 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dae85-a03c-4180-a2b3-e857df1401e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.490706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dae85-a03c-4180-a2b3-e857df1401e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.491050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dae85-a03c-4180-a2b3-e857df1401e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.491847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443dae85-a03c-4180-a2b3-e857df1401e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:09.493528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443dae85-a03c-4180-a2b3-e857df1401e0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:09.504150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1266/2518] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:09.508260 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a95f1-d456-4da9-96be-48e37004e535 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:09.520614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a95f1-d456-4da9-96be-48e37004e535 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:09.525678 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4e9bf-ee4c-4330-bf89-7d069778f4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:09.526543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4e9bf-ee4c-4330-bf89-7d069778f4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.526861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4e9bf-ee4c-4330-bf89-7d069778f4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.527803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4e9bf-ee4c-4330-bf89-7d069778f4f7 admin admin] PATH_INFO: `/v3/projects/e58d2ae3ee1b45cbac9cbb69bc198031/users/ea2d1d60221d4d98a5baccc7d53db654/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.545289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4e9bf-ee4c-4330-bf89-7d069778f4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58d2ae3ee1b45cbac9cbb69bc198031, user_id=ea2d1d60221d4d98a5baccc7d53db654, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:09.548254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4e9bf-ee4c-4330-bf89-7d069778f4f7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:09.616370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1253/2519] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:09 2022] PUT /identity/v3/projects/e58d2ae3ee1b45cbac9cbb69bc198031/users/ea2d1d60221d4d98a5baccc7d53db654/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:36:09.631861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d7ec1-9cbf-4c2b-a5a8-ab3f0a0d3536 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.632159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d7ec1-9cbf-4c2b-a5a8-ab3f0a0d3536 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.632445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d7ec1-9cbf-4c2b-a5a8-ab3f0a0d3536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.660571 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6b1d7ec1-9cbf-4c2b-a5a8-ab3f0a0d3536 None None] MFA Rules not processed for user `ea2d1d60221d4d98a5baccc7d53db654`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:09.661932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6b1d7ec1-9cbf-4c2b-a5a8-ab3f0a0d3536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:09.674014 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:09.727525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1267/2520] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:09.739524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4e9bf-ee4c-4330-bf89-7d069778f4f7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:09.741217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ade4e9bf-ee4c-4330-bf89-7d069778f4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:09.847712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4e9bf-ee4c-4330-bf89-7d069778f4f7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:09.853893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef68de2-c134-476f-a484-1000a78a72c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:09.854583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef68de2-c134-476f-a484-1000a78a72c1 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:09.855049 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef68de2-c134-476f-a484-1000a78a72c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:09.855321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef68de2-c134-476f-a484-1000a78a72c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:09.863852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef68de2-c134-476f-a484-1000a78a72c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:09.865479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef68de2-c134-476f-a484-1000a78a72c1 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:09.875948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1254/2521] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:10.090474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d7ec1-9cbf-4c2b-a5a8-ab3f0a0d3536 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:10.152501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d7ec1-9cbf-4c2b-a5a8-ab3f0a0d3536 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:10.157061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:10.157937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:10.158223 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:10.158533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] PATH_INFO: `/v3/users/ea2d1d60221d4d98a5baccc7d53db654` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:10.159852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2d1d60221d4d98a5baccc7d53db654)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:10.161905 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:10.407376 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2d1d60221d4d98a5baccc7d53db654'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:10.414317 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2d1d60221d4d98a5baccc7d53db654'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:10.415045 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2d1d60221d4d98a5baccc7d53db654'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:10.415645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2d1d60221d4d98a5baccc7d53db654'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:10.461230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2d1d60221d4d98a5baccc7d53db654'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:10.479389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1268/2522] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:10 2022] DELETE /identity/v3/users/ea2d1d60221d4d98a5baccc7d53db654 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:10.484340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9accfa-7fc4-4bf9-8035-5f90a26e920d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:10.484925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9accfa-7fc4-4bf9-8035-5f90a26e920d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:10.484925 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9accfa-7fc4-4bf9-8035-5f90a26e920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:10.490801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:10.504332 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fb9accfa-7fc4-4bf9-8035-5f90a26e920d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:10.507532 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fb9accfa-7fc4-4bf9-8035-5f90a26e920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:10.570382 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1255/2523] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:10.580980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:10.582429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:10.618119 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:10.641538 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc620a7-2d96-48a7-a2aa-4a1e5ce84229 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:10.645157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a38fd-1f41-4f0e-bd17-fe0201f72dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:10.645823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a38fd-1f41-4f0e-bd17-fe0201f72dcd service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:10.646103 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a38fd-1f41-4f0e-bd17-fe0201f72dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:10.646375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a38fd-1f41-4f0e-bd17-fe0201f72dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:10.655961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a38fd-1f41-4f0e-bd17-fe0201f72dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:10.657434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a38fd-1f41-4f0e-bd17-fe0201f72dcd service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:10.667096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1269/2524] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:11.080698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9accfa-7fc4-4bf9-8035-5f90a26e920d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:11.082337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fb9accfa-7fc4-4bf9-8035-5f90a26e920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:11.095847 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9accfa-7fc4-4bf9-8035-5f90a26e920d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:11.102012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:11.103107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:11.103398 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:11.103720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] PATH_INFO: `/v3/projects/e58d2ae3ee1b45cbac9cbb69bc198031` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:11.104895 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e58d2ae3ee1b45cbac9cbb69bc198031)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:11.107639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:11.257473 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e58d2ae3ee1b45cbac9cbb69bc198031', 'request_id': 'req-975467a3-1197-4da6-9fb3-1621debf2317'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:11.264138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e58d2ae3ee1b45cbac9cbb69bc198031', 'request_id': 'req-975467a3-1197-4da6-9fb3-1621debf2317'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:11.341768 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e58d2ae3ee1b45cbac9cbb69bc198031', 'request_id': 'req-975467a3-1197-4da6-9fb3-1621debf2317'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:11.353507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] The token cache is being invalidate because project e58d2ae3ee1b45cbac9cbb69bc198031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:11.353794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:11.355037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-975467a3-1197-4da6-9fb3-1621debf2317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:11.357385 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1256/2525] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:11 2022] DELETE /identity/v3/projects/e58d2ae3ee1b45cbac9cbb69bc198031 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:11.362222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b5df-da3d-4c35-9353-481b101c4d7b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:11.362560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b5df-da3d-4c35-9353-481b101c4d7b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:11.362838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b5df-da3d-4c35-9353-481b101c4d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:11.380274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6298b5df-da3d-4c35-9353-481b101c4d7b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:11.381479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6298b5df-da3d-4c35-9353-481b101c4d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:11.439982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1270/2526] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:11.448734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f7410-8a8d-49e9-a98c-fe22241e8896 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:11.449034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f7410-8a8d-49e9-a98c-fe22241e8896 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:11.449345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f7410-8a8d-49e9-a98c-fe22241e8896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:11.468294 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-155f7410-8a8d-49e9-a98c-fe22241e8896 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:11.469500 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-155f7410-8a8d-49e9-a98c-fe22241e8896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:11.477429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1257/2527] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:11.482306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298b5df-da3d-4c35-9353-481b101c4d7b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:11.492061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298b5df-da3d-4c35-9353-481b101c4d7b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:11.496642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd03d22-068e-4080-8744-1b34e9df37db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:11.497374 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd03d22-068e-4080-8744-1b34e9df37db admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:11.497662 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd03d22-068e-4080-8744-1b34e9df37db admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:11.497943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd03d22-068e-4080-8744-1b34e9df37db admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:11.498707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd03d22-068e-4080-8744-1b34e9df37db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:36:11.499022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd03d22-068e-4080-8744-1b34e9df37db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:11.500652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd03d22-068e-4080-8744-1b34e9df37db admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:11.514664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1271/2528] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:36:11.519071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f7410-8a8d-49e9-a98c-fe22241e8896 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:11.530555 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f7410-8a8d-49e9-a98c-fe22241e8896 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:11.535209 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf6f49-cdd7-4623-b334-793b70d0608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:11.535864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf6f49-cdd7-4623-b334-793b70d0608b admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:11.536150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf6f49-cdd7-4623-b334-793b70d0608b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:11.536424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf6f49-cdd7-4623-b334-793b70d0608b admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:11.537121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf6f49-cdd7-4623-b334-793b70d0608b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:11.539052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf6f49-cdd7-4623-b334-793b70d0608b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:11.623151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1258/2529] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:11 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:11.627441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd03d22-068e-4080-8744-1b34e9df37db admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:11.638300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd03d22-068e-4080-8744-1b34e9df37db admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:11.698556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f739c-83c3-4bb3-a4b4-f191cd722d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:11.699234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f739c-83c3-4bb3-a4b4-f191cd722d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:11.699563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f739c-83c3-4bb3-a4b4-f191cd722d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:11.699964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f739c-83c3-4bb3-a4b4-f191cd722d47 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:11.700667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f739c-83c3-4bb3-a4b4-f191cd722d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:11.702685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f739c-83c3-4bb3-a4b4-f191cd722d47 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:11.821954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1272/2530] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:11 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:11.826152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf6f49-cdd7-4623-b334-793b70d0608b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:11.836330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf6f49-cdd7-4623-b334-793b70d0608b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:11.841003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e13ce-6523-4098-a304-7e929662da82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:11.841651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e13ce-6523-4098-a304-7e929662da82 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:11.841933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e13ce-6523-4098-a304-7e929662da82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:11.842212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e13ce-6523-4098-a304-7e929662da82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:11.843010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e13ce-6523-4098-a304-7e929662da82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:11.844603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e13ce-6523-4098-a304-7e929662da82 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:11.856520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1259/2531] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:11.862160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f739c-83c3-4bb3-a4b4-f191cd722d47 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:11.873535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f739c-83c3-4bb3-a4b4-f191cd722d47 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:11.878450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad7444-dcb8-4b99-9ee9-1c5f62f5f113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:11.879257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad7444-dcb8-4b99-9ee9-1c5f62f5f113 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:11.879449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad7444-dcb8-4b99-9ee9-1c5f62f5f113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:11.879726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad7444-dcb8-4b99-9ee9-1c5f62f5f113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:11.880462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad7444-dcb8-4b99-9ee9-1c5f62f5f113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:11.882053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad7444-dcb8-4b99-9ee9-1c5f62f5f113 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:11.897284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1273/2532] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:11.901160 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e13ce-6523-4098-a304-7e929662da82 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:11.914748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e13ce-6523-4098-a304-7e929662da82 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:11.920964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1351409-18dd-4913-bc70-6b8135888039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:11.921878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1351409-18dd-4913-bc70-6b8135888039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:11.922458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1351409-18dd-4913-bc70-6b8135888039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:11.922747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1351409-18dd-4913-bc70-6b8135888039 admin admin] PATH_INFO: `/v3/projects/52526ffcefdf4f1fb7791e24d0411b6e/users/12f7ad0b989747189d4d3a5bf653590c/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:11.943062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1351409-18dd-4913-bc70-6b8135888039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52526ffcefdf4f1fb7791e24d0411b6e, user_id=12f7ad0b989747189d4d3a5bf653590c, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:11.945747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1351409-18dd-4913-bc70-6b8135888039 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:12.016773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1260/2533] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:11 2022] PUT /identity/v3/projects/52526ffcefdf4f1fb7791e24d0411b6e/users/12f7ad0b989747189d4d3a5bf653590c/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:12.029757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea107ee2-8bd0-42ca-8f36-d31aa143a8f3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:12.030050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea107ee2-8bd0-42ca-8f36-d31aa143a8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:12.030330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea107ee2-8bd0-42ca-8f36-d31aa143a8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:12.057806 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ea107ee2-8bd0-42ca-8f36-d31aa143a8f3 None None] MFA Rules not processed for user `12f7ad0b989747189d4d3a5bf653590c`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:12.059408 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ea107ee2-8bd0-42ca-8f36-d31aa143a8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:12.092197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:12.126475 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1274/2534] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:12.130998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1351409-18dd-4913-bc70-6b8135888039 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:12.157115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:12.192899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1351409-18dd-4913-bc70-6b8135888039 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:12.197507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62936a67-8a43-4dc9-95e6-938f53f71494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:12.198165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62936a67-8a43-4dc9-95e6-938f53f71494 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:12.198477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62936a67-8a43-4dc9-95e6-938f53f71494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:12.198762 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62936a67-8a43-4dc9-95e6-938f53f71494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:12.199484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62936a67-8a43-4dc9-95e6-938f53f71494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:12.201425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62936a67-8a43-4dc9-95e6-938f53f71494 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:12.264319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1261/2535] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:12 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:12.268348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea107ee2-8bd0-42ca-8f36-d31aa143a8f3 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:12.280005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea107ee2-8bd0-42ca-8f36-d31aa143a8f3 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:12.336956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db95ddf-d0d8-4a8c-be63-d317a84cf7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:12.337546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db95ddf-d0d8-4a8c-be63-d317a84cf7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:12.337835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db95ddf-d0d8-4a8c-be63-d317a84cf7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:12.338114 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db95ddf-d0d8-4a8c-be63-d317a84cf7e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:12.338842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db95ddf-d0d8-4a8c-be63-d317a84cf7e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:12.341462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db95ddf-d0d8-4a8c-be63-d317a84cf7e1 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:12.416393 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1275/2536] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:12 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:12.420559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62936a67-8a43-4dc9-95e6-938f53f71494 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:12.432831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62936a67-8a43-4dc9-95e6-938f53f71494 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:12.438073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9a06-6442-48d9-9070-b01d7ff5216c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:12.438778 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9a06-6442-48d9-9070-b01d7ff5216c admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:12.439065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9a06-6442-48d9-9070-b01d7ff5216c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:12.439341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b9a06-6442-48d9-9070-b01d7ff5216c admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:12.440075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9a06-6442-48d9-9070-b01d7ff5216c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:12.441668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b9a06-6442-48d9-9070-b01d7ff5216c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:12.453261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1262/2537] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:12.457449 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db95ddf-d0d8-4a8c-be63-d317a84cf7e1 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:12.468422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db95ddf-d0d8-4a8c-be63-d317a84cf7e1 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:12.473305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29554d1e-0742-46f4-b373-ba0a9a0d3108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:12.474179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29554d1e-0742-46f4-b373-ba0a9a0d3108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:12.474497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29554d1e-0742-46f4-b373-ba0a9a0d3108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:12.474774 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29554d1e-0742-46f4-b373-ba0a9a0d3108 admin admin] PATH_INFO: `/v3/projects/42c2ce52e05a417098f0191efb125878/users/f7baaadaaf344e62be6d8d5977074dac/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:12.490011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29554d1e-0742-46f4-b373-ba0a9a0d3108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42c2ce52e05a417098f0191efb125878, user_id=f7baaadaaf344e62be6d8d5977074dac, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:12.492727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29554d1e-0742-46f4-b373-ba0a9a0d3108 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:12.561385 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1276/2538] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:12 2022] PUT /identity/v3/projects/42c2ce52e05a417098f0191efb125878/users/f7baaadaaf344e62be6d8d5977074dac/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:12.565507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9a06-6442-48d9-9070-b01d7ff5216c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:12.576198 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b9a06-6442-48d9-9070-b01d7ff5216c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:12.632893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86109708-16a5-49de-9dee-fd19fa37e80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:12.633554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86109708-16a5-49de-9dee-fd19fa37e80a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:12.633843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86109708-16a5-49de-9dee-fd19fa37e80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:12.634123 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86109708-16a5-49de-9dee-fd19fa37e80a admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:12.634884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86109708-16a5-49de-9dee-fd19fa37e80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:12.636479 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86109708-16a5-49de-9dee-fd19fa37e80a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:12.646945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1263/2539] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:12.651365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29554d1e-0742-46f4-b373-ba0a9a0d3108 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:12.661840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29554d1e-0742-46f4-b373-ba0a9a0d3108 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:12.666541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f47d6-9b67-418b-9052-96bf66eb5168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:12.667420 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f47d6-9b67-418b-9052-96bf66eb5168 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:12.667707 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f47d6-9b67-418b-9052-96bf66eb5168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:12.667985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f47d6-9b67-418b-9052-96bf66eb5168 admin admin] PATH_INFO: `/v3/projects/42c2ce52e05a417098f0191efb125878/users/f7baaadaaf344e62be6d8d5977074dac/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:12.670099 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f47d6-9b67-418b-9052-96bf66eb5168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42c2ce52e05a417098f0191efb125878, user_id=f7baaadaaf344e62be6d8d5977074dac, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:12.672794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f47d6-9b67-418b-9052-96bf66eb5168 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:12.683506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1277/2540] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:12 2022] PUT /identity/v3/projects/42c2ce52e05a417098f0191efb125878/users/f7baaadaaf344e62be6d8d5977074dac/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:12.694143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39af53-08e8-4c49-9874-0d40eeb91320 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:12.697600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39af53-08e8-4c49-9874-0d40eeb91320 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:12.697600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39af53-08e8-4c49-9874-0d40eeb91320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:12.722314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-eb39af53-08e8-4c49-9874-0d40eeb91320 None None] MFA Rules not processed for user `f7baaadaaf344e62be6d8d5977074dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:12.723556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-eb39af53-08e8-4c49-9874-0d40eeb91320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:12.792752 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1264/2541] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:12.805068 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f47d6-9b67-418b-9052-96bf66eb5168 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:12.807050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2f3f47d6-9b67-418b-9052-96bf66eb5168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:12.877513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f47d6-9b67-418b-9052-96bf66eb5168 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:12.881132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441d4ea-dad2-4b34-a118-f63009e84922 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:12.881772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441d4ea-dad2-4b34-a118-f63009e84922 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:12.882051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441d4ea-dad2-4b34-a118-f63009e84922 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:12.882325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441d4ea-dad2-4b34-a118-f63009e84922 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:12.893323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441d4ea-dad2-4b34-a118-f63009e84922 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:12.894686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441d4ea-dad2-4b34-a118-f63009e84922 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:12.904821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1278/2542] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:13.913139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39af53-08e8-4c49-9874-0d40eeb91320 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:13.933261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:13.975846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39af53-08e8-4c49-9874-0d40eeb91320 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:13.981249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:13.982127 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:13.982431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:13.982718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] PATH_INFO: `/v3/users/12f7ad0b989747189d4d3a5bf653590c` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:13.983929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f7ad0b989747189d4d3a5bf653590c)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:13.985963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:14.162046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f7ad0b989747189d4d3a5bf653590c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:14.162716 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f7ad0b989747189d4d3a5bf653590c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:14.176358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f7ad0b989747189d4d3a5bf653590c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:14.177044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f7ad0b989747189d4d3a5bf653590c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:14.233362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f7ad0b989747189d4d3a5bf653590c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:14.250621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1265/2543] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:13 2022] DELETE /identity/v3/users/12f7ad0b989747189d4d3a5bf653590c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:14.254850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9014267-5e92-4edd-ad8c-03c41aace017 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:14.255365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9014267-5e92-4edd-ad8c-03c41aace017 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:14.255590 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9014267-5e92-4edd-ad8c-03c41aace017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:14.272827 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c9014267-5e92-4edd-ad8c-03c41aace017 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:14.274024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c9014267-5e92-4edd-ad8c-03c41aace017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:14.284745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:14.334272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1279/2544] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:14.344711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:14.346303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:14.402465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89363fd-7792-454d-a64d-41d579468874 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:14.406918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c1753-512b-4a00-8e3f-dc8c8d22c284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:14.407557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c1753-512b-4a00-8e3f-dc8c8d22c284 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:14.407890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c1753-512b-4a00-8e3f-dc8c8d22c284 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:14.408164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c1753-512b-4a00-8e3f-dc8c8d22c284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:14.417194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c1753-512b-4a00-8e3f-dc8c8d22c284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:14.419468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c1753-512b-4a00-8e3f-dc8c8d22c284 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:14.429070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1266/2545] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:14.863958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9014267-5e92-4edd-ad8c-03c41aace017 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:14.865464 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c9014267-5e92-4edd-ad8c-03c41aace017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:14.884133 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9014267-5e92-4edd-ad8c-03c41aace017 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:14.894684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:14.896000 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:14.896356 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:14.896527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] PATH_INFO: `/v3/projects/52526ffcefdf4f1fb7791e24d0411b6e` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:14.897817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52526ffcefdf4f1fb7791e24d0411b6e)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:14.902258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:15.029497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52526ffcefdf4f1fb7791e24d0411b6e', 'request_id': 'req-28464248-48ab-4401-b1c9-3b79e23d8842'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.035142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52526ffcefdf4f1fb7791e24d0411b6e', 'request_id': 'req-28464248-48ab-4401-b1c9-3b79e23d8842'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.118591 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52526ffcefdf4f1fb7791e24d0411b6e', 'request_id': 'req-28464248-48ab-4401-b1c9-3b79e23d8842'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.131683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] The token cache is being invalidate because project 52526ffcefdf4f1fb7791e24d0411b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:15.131977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.132691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.134446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1280/2546] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:14 2022] DELETE /identity/v3/projects/52526ffcefdf4f1fb7791e24d0411b6e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:15.139289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c1753-512b-4a00-8e3f-dc8c8d22c284 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:15.149099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c1753-512b-4a00-8e3f-dc8c8d22c284 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:15.186417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:15.205088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:15.205962 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:15.206240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:15.206549 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] PATH_INFO: `/v3/users/f7baaadaaf344e62be6d8d5977074dac` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:15.207729 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7baaadaaf344e62be6d8d5977074dac)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:15.209699 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:15.463504 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7baaadaaf344e62be6d8d5977074dac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.464147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7baaadaaf344e62be6d8d5977074dac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.476151 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7baaadaaf344e62be6d8d5977074dac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.476878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7baaadaaf344e62be6d8d5977074dac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.547269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b7e597b6-e623-47da-9bbf-5bde0a08b760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7baaadaaf344e62be6d8d5977074dac'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:15.565803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1267/2547] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:15 2022] DELETE /identity/v3/users/f7baaadaaf344e62be6d8d5977074dac => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:16.195503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:16.197339 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:16.215963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28464248-48ab-4401-b1c9-3b79e23d8842 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:16.276094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:16.276883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:16.277285 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:16.277423 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] PATH_INFO: `/v3/projects/42c2ce52e05a417098f0191efb125878` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:16.279300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=42c2ce52e05a417098f0191efb125878)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:16.281413 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:16.405096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:16.410997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42c2ce52e05a417098f0191efb125878', 'request_id': 'req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.416430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42c2ce52e05a417098f0191efb125878', 'request_id': 'req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.484205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42c2ce52e05a417098f0191efb125878', 'request_id': 'req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.495809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] The token cache is being invalidate because project 42c2ce52e05a417098f0191efb125878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:16.496101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.496757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-fb8f1d6c-f367-4467-b5a8-7fbd61269bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.498857 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1281/2548] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:16 2022] DELETE /identity/v3/projects/42c2ce52e05a417098f0191efb125878 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:16.503843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf21bc-27cf-422a-894f-1374a8700814 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:16.504361 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf21bc-27cf-422a-894f-1374a8700814 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:16.504463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf21bc-27cf-422a-894f-1374a8700814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:16.522918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-52cf21bc-27cf-422a-894f-1374a8700814 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:16.524131 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-52cf21bc-27cf-422a-894f-1374a8700814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:16.597333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1268/2549] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:16.610999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bd110-bb0d-4249-adf0-8ef9f7bd1b10 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:16.610999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bd110-bb0d-4249-adf0-8ef9f7bd1b10 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:16.610999 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bd110-bb0d-4249-adf0-8ef9f7bd1b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:16.613727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf21bc-27cf-422a-894f-1374a8700814 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:16.615120 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-52cf21bc-27cf-422a-894f-1374a8700814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:16.626943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f12bd110-bb0d-4249-adf0-8ef9f7bd1b10 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:16.628263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf21bc-27cf-422a-894f-1374a8700814 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:16.628380 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f12bd110-bb0d-4249-adf0-8ef9f7bd1b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:16.631869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:16.632777 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:16.633032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:16.633311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] PATH_INFO: `/v3/users/95f4e782ac72403c981a0daa2406973d` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:16.634605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] RBAC: Authorizing `identity:delete_user(user_id=95f4e782ac72403c981a0daa2406973d)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:16.636235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1282/2550] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:16.639999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:16.640569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bd110-bb0d-4249-adf0-8ef9f7bd1b10 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:16.649850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bd110-bb0d-4249-adf0-8ef9f7bd1b10 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:16.656778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231988f6-5b24-4d48-84f1-1b446f522666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:16.657004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231988f6-5b24-4d48-84f1-1b446f522666 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:16.657312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231988f6-5b24-4d48-84f1-1b446f522666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:16.657589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231988f6-5b24-4d48-84f1-1b446f522666 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:16.658319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231988f6-5b24-4d48-84f1-1b446f522666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:36:16.658684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231988f6-5b24-4d48-84f1-1b446f522666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:16.660301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231988f6-5b24-4d48-84f1-1b446f522666 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:16.674041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1283/2551] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:36:16.678273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231988f6-5b24-4d48-84f1-1b446f522666 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:16.688460 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231988f6-5b24-4d48-84f1-1b446f522666 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:16.693106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811824d-f019-4d14-815b-1ac04f810f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:16.693741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811824d-f019-4d14-815b-1ac04f810f50 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:16.694127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811824d-f019-4d14-815b-1ac04f810f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:16.694482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811824d-f019-4d14-815b-1ac04f810f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:16.695177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811824d-f019-4d14-815b-1ac04f810f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:16.697071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811824d-f019-4d14-815b-1ac04f810f50 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:16.845589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1284/2552] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:16 2022] POST /identity/v3/projects => generated 391 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:16.845589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811824d-f019-4d14-815b-1ac04f810f50 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:16.882681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95f4e782ac72403c981a0daa2406973d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.883420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95f4e782ac72403c981a0daa2406973d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.889818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:16.897274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95f4e782ac72403c981a0daa2406973d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.898600 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95f4e782ac72403c981a0daa2406973d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.920765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811824d-f019-4d14-815b-1ac04f810f50 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:16.926802 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d22ac5-846b-4625-b4f9-181ae3ef2a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:16.927558 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d22ac5-846b-4625-b4f9-181ae3ef2a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:16.927843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d22ac5-846b-4625-b4f9-181ae3ef2a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:16.928115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d22ac5-846b-4625-b4f9-181ae3ef2a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:16.928848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d22ac5-846b-4625-b4f9-181ae3ef2a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:16.931271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d22ac5-846b-4625-b4f9-181ae3ef2a81 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:16.970114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-fd089fa3-3b94-4660-9797-be860c0d126a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95f4e782ac72403c981a0daa2406973d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:16.983737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1269/2553] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:16 2022] DELETE /identity/v3/users/95f4e782ac72403c981a0daa2406973d => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:36:16.988372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17426ecb-a0b2-48ad-8071-88eb04b4a55b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:16.988742 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17426ecb-a0b2-48ad-8071-88eb04b4a55b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:16.989100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17426ecb-a0b2-48ad-8071-88eb04b4a55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.000844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1285/2554] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:16 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:17.005948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d22ac5-846b-4625-b4f9-181ae3ef2a81 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.007904 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d1d22ac5-846b-4625-b4f9-181ae3ef2a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:17.010059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-17426ecb-a0b2-48ad-8071-88eb04b4a55b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:17.011356 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-17426ecb-a0b2-48ad-8071-88eb04b4a55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:17.033209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d22ac5-846b-4625-b4f9-181ae3ef2a81 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:17.085864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1270/2555] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:17.100261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17426ecb-a0b2-48ad-8071-88eb04b4a55b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.101865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-17426ecb-a0b2-48ad-8071-88eb04b4a55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:17.111742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408929-ba8f-4feb-ad4b-a0b79d2c199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:17.112425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408929-ba8f-4feb-ad4b-a0b79d2c199c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.113060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408929-ba8f-4feb-ad4b-a0b79d2c199c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.113060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408929-ba8f-4feb-ad4b-a0b79d2c199c admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.113750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a408929-ba8f-4feb-ad4b-a0b79d2c199c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:17.115406 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a408929-ba8f-4feb-ad4b-a0b79d2c199c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:17.128946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1286/2556] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:17.130585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408929-ba8f-4feb-ad4b-a0b79d2c199c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.143217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408929-ba8f-4feb-ad4b-a0b79d2c199c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:17.149930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3babfa-b153-46c1-b9ae-5ca43e60ec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:17.151038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3babfa-b153-46c1-b9ae-5ca43e60ec07 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.151366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3babfa-b153-46c1-b9ae-5ca43e60ec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.151650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3babfa-b153-46c1-b9ae-5ca43e60ec07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.152371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3babfa-b153-46c1-b9ae-5ca43e60ec07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:17.154041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3babfa-b153-46c1-b9ae-5ca43e60ec07 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:17.163732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17426ecb-a0b2-48ad-8071-88eb04b4a55b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:17.164395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1287/2557] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:17.168112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbc786-f8e2-43e1-953e-2ddef574ff0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:17.168576 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3babfa-b153-46c1-b9ae-5ca43e60ec07 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.169355 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbc786-f8e2-43e1-953e-2ddef574ff0a service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.169737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbc786-f8e2-43e1-953e-2ddef574ff0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.169954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbc786-f8e2-43e1-953e-2ddef574ff0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.173685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:17.179570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3babfa-b153-46c1-b9ae-5ca43e60ec07 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:17.181302 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbc786-f8e2-43e1-953e-2ddef574ff0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:17.182864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbc786-f8e2-43e1-953e-2ddef574ff0a service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:17.187758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9ff2c-42fc-4faf-9d97-c251ac1fd437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:17.191215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9ff2c-42fc-4faf-9d97-c251ac1fd437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.191215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9ff2c-42fc-4faf-9d97-c251ac1fd437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.191215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9ff2c-42fc-4faf-9d97-c251ac1fd437 admin admin] PATH_INFO: `/v3/projects/ac5fcad06279414e868008a595e83fe2/users/051eb8ce715d4466a50beee711e0f303/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.195494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1271/2558] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:36:17.206653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9ff2c-42fc-4faf-9d97-c251ac1fd437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5fcad06279414e868008a595e83fe2, user_id=051eb8ce715d4466a50beee711e0f303, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:17.209378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9ff2c-42fc-4faf-9d97-c251ac1fd437 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:17.286391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1288/2559] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:17 2022] PUT /identity/v3/projects/ac5fcad06279414e868008a595e83fe2/users/051eb8ce715d4466a50beee711e0f303/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:36:17.299929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90691f-45da-4998-b36f-ae30e8e3ef0f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.300377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90691f-45da-4998-b36f-ae30e8e3ef0f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.300587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90691f-45da-4998-b36f-ae30e8e3ef0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.332526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1e90691f-45da-4998-b36f-ae30e8e3ef0f None None] MFA Rules not processed for user `051eb8ce715d4466a50beee711e0f303`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:17.333769 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1e90691f-45da-4998-b36f-ae30e8e3ef0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:17.404980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1272/2560] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:17.409696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9ff2c-42fc-4faf-9d97-c251ac1fd437 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.478004 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9ff2c-42fc-4faf-9d97-c251ac1fd437 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:17.483067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f9752-5c07-4d7c-bcd6-0c6da8434f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:17.483730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f9752-5c07-4d7c-bcd6-0c6da8434f60 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.484040 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f9752-5c07-4d7c-bcd6-0c6da8434f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.484316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f9752-5c07-4d7c-bcd6-0c6da8434f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.485176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f9752-5c07-4d7c-bcd6-0c6da8434f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:17.487018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f9752-5c07-4d7c-bcd6-0c6da8434f60 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:17.566988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1289/2561] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:17 2022] POST /identity/v3/projects => generated 391 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:17.570993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90691f-45da-4998-b36f-ae30e8e3ef0f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.578535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:17.582843 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90691f-45da-4998-b36f-ae30e8e3ef0f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:17.642632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b75caa-53ab-47fb-a7eb-9670e070bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:17.643626 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b75caa-53ab-47fb-a7eb-9670e070bde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.643901 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b75caa-53ab-47fb-a7eb-9670e070bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.644175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b75caa-53ab-47fb-a7eb-9670e070bde1 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.644879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b75caa-53ab-47fb-a7eb-9670e070bde1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:17.646906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b75caa-53ab-47fb-a7eb-9670e070bde1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:17.749144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f9752-5c07-4d7c-bcd6-0c6da8434f60 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.751011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ce6f9752-5c07-4d7c-bcd6-0c6da8434f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:17.773584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1273/2562] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:17 2022] POST /identity/v3/users => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:17.777953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b75caa-53ab-47fb-a7eb-9670e070bde1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.780552 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f9752-5c07-4d7c-bcd6-0c6da8434f60 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:17.785642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:17.787690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.788329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.788722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] PATH_INFO: `/v3/projects/7ff23bff1a234db08b13c7b93127f279` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.790157 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff23bff1a234db08b13c7b93127f279)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:17.793597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:17.793703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b75caa-53ab-47fb-a7eb-9670e070bde1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:17.798976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036756a6-3ff6-45d6-8c06-21368a339982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:17.799857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036756a6-3ff6-45d6-8c06-21368a339982 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.800083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036756a6-3ff6-45d6-8c06-21368a339982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.800676 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036756a6-3ff6-45d6-8c06-21368a339982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.801234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036756a6-3ff6-45d6-8c06-21368a339982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:17.802974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036756a6-3ff6-45d6-8c06-21368a339982 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:17.814256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1274/2563] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:17.818259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036756a6-3ff6-45d6-8c06-21368a339982 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.829496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036756a6-3ff6-45d6-8c06-21368a339982 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:17.834465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf96acd-9581-47dc-beb9-1c1e80c2496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:17.835366 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf96acd-9581-47dc-beb9-1c1e80c2496b admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:17.835649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf96acd-9581-47dc-beb9-1c1e80c2496b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:17.835928 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf96acd-9581-47dc-beb9-1c1e80c2496b admin admin] PATH_INFO: `/v3/projects/befc5073e6094c0a8ca911cf5e0dff00/users/8dc4b1816c2f45b8bf74b5486648deab/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:17.852640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf96acd-9581-47dc-beb9-1c1e80c2496b admin admin] RBAC: Authorizing `identity:create_grant(project_id=befc5073e6094c0a8ca911cf5e0dff00, user_id=8dc4b1816c2f45b8bf74b5486648deab, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:17.855380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf96acd-9581-47dc-beb9-1c1e80c2496b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:17.920568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1275/2564] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:17 2022] PUT /identity/v3/projects/befc5073e6094c0a8ca911cf5e0dff00/users/8dc4b1816c2f45b8bf74b5486648deab/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:36:17.924345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf96acd-9581-47dc-beb9-1c1e80c2496b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:17.928088 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff23bff1a234db08b13c7b93127f279', 'request_id': 'req-ac28b83f-f48d-4409-878f-ee769acc4bde'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:17.934092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff23bff1a234db08b13c7b93127f279', 'request_id': 'req-ac28b83f-f48d-4409-878f-ee769acc4bde'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:17.987876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf96acd-9581-47dc-beb9-1c1e80c2496b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.015399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff23bff1a234db08b13c7b93127f279', 'request_id': 'req-ac28b83f-f48d-4409-878f-ee769acc4bde'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:18.029075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] The token cache is being invalidate because project 7ff23bff1a234db08b13c7b93127f279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:18.029568 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:18.030334 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-ac28b83f-f48d-4409-878f-ee769acc4bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:18.031991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1290/2565] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:17 2022] DELETE /identity/v3/projects/7ff23bff1a234db08b13c7b93127f279 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:18.036978 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e433-563a-422e-a93e-03d517afa8f3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.042909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e433-563a-422e-a93e-03d517afa8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.042909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881e433-563a-422e-a93e-03d517afa8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.051442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb8335-8b18-4e12-8157-937e89a5828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.052112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb8335-8b18-4e12-8157-937e89a5828b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.052425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb8335-8b18-4e12-8157-937e89a5828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.052698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb8335-8b18-4e12-8157-937e89a5828b admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.053433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb8335-8b18-4e12-8157-937e89a5828b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.055124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb8335-8b18-4e12-8157-937e89a5828b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.060123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c881e433-563a-422e-a93e-03d517afa8f3 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:18.061475 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c881e433-563a-422e-a93e-03d517afa8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:18.065231 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1276/2566] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:18.069180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb8335-8b18-4e12-8157-937e89a5828b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.069886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1291/2567] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:18.071023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-49eb8335-8b18-4e12-8157-937e89a5828b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:18.078003 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bf1b0-6866-4134-9a19-618c02323a6a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.078298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bf1b0-6866-4134-9a19-618c02323a6a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.078615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bf1b0-6866-4134-9a19-618c02323a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.083440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb8335-8b18-4e12-8157-937e89a5828b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.088199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23304c4a-d753-4fc6-81f9-f5a4833fa880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.089158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23304c4a-d753-4fc6-81f9-f5a4833fa880 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.089367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23304c4a-d753-4fc6-81f9-f5a4833fa880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.089645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23304c4a-d753-4fc6-81f9-f5a4833fa880 admin admin] PATH_INFO: `/v3/projects/befc5073e6094c0a8ca911cf5e0dff00/users/8dc4b1816c2f45b8bf74b5486648deab/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.092995 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23304c4a-d753-4fc6-81f9-f5a4833fa880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=befc5073e6094c0a8ca911cf5e0dff00, user_id=8dc4b1816c2f45b8bf74b5486648deab, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.095846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23304c4a-d753-4fc6-81f9-f5a4833fa880 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.097775 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9a2bf1b0-6866-4134-9a19-618c02323a6a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:18.098998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9a2bf1b0-6866-4134-9a19-618c02323a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:18.106455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1277/2568] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:18 2022] PUT /identity/v3/projects/befc5073e6094c0a8ca911cf5e0dff00/users/8dc4b1816c2f45b8bf74b5486648deab/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:36:18.108647 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:18.117567 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9b9f8-cce2-41e7-9b3e-cb45f1fd6b11 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.117858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9b9f8-cce2-41e7-9b3e-cb45f1fd6b11 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.118137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9b9f8-cce2-41e7-9b3e-cb45f1fd6b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.137440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:18.153402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-51a9b9f8-cce2-41e7-9b3e-cb45f1fd6b11 None None] MFA Rules not processed for user `8dc4b1816c2f45b8bf74b5486648deab`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:18.155909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-51a9b9f8-cce2-41e7-9b3e-cb45f1fd6b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:18.164804 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1292/2569] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:18.168599 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bf1b0-6866-4134-9a19-618c02323a6a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.178646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bf1b0-6866-4134-9a19-618c02323a6a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.183673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3505e7a-2184-4fc1-8407-8ca750eceafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.184328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3505e7a-2184-4fc1-8407-8ca750eceafd admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.184615 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3505e7a-2184-4fc1-8407-8ca750eceafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.184976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3505e7a-2184-4fc1-8407-8ca750eceafd admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.185709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3505e7a-2184-4fc1-8407-8ca750eceafd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:36:18.186030 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3505e7a-2184-4fc1-8407-8ca750eceafd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.187692 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3505e7a-2184-4fc1-8407-8ca750eceafd admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.199400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1293/2570] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:36:18.204758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3505e7a-2184-4fc1-8407-8ca750eceafd admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.214834 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3505e7a-2184-4fc1-8407-8ca750eceafd admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.220360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656713-6584-4888-bba2-1a5388877ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.221011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656713-6584-4888-bba2-1a5388877ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.221299 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656713-6584-4888-bba2-1a5388877ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.221582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656713-6584-4888-bba2-1a5388877ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.222274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d656713-6584-4888-bba2-1a5388877ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.224226 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d656713-6584-4888-bba2-1a5388877ef7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.231859 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1278/2571] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:18.243189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9b9f8-cce2-41e7-9b3e-cb45f1fd6b11 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.244547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-51a9b9f8-cce2-41e7-9b3e-cb45f1fd6b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:18.307371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9b9f8-cce2-41e7-9b3e-cb45f1fd6b11 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.311773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe80d0-ab94-4d57-8c4f-e9d466df8fa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.315003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe80d0-ab94-4d57-8c4f-e9d466df8fa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.315003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe80d0-ab94-4d57-8c4f-e9d466df8fa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.315003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe80d0-ab94-4d57-8c4f-e9d466df8fa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.323672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe80d0-ab94-4d57-8c4f-e9d466df8fa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.325124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe80d0-ab94-4d57-8c4f-e9d466df8fa9 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.335591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1279/2572] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:36:18.343973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1294/2573] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:18 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:18.347725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe80d0-ab94-4d57-8c4f-e9d466df8fa9 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.358829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe80d0-ab94-4d57-8c4f-e9d466df8fa9 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.383847 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656713-6584-4888-bba2-1a5388877ef7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.385595 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6d656713-6584-4888-bba2-1a5388877ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:18.420880 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7629e-2f80-4e0c-8fc5-68c9e34d323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.421547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7629e-2f80-4e0c-8fc5-68c9e34d323e admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.421829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7629e-2f80-4e0c-8fc5-68c9e34d323e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.422102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7629e-2f80-4e0c-8fc5-68c9e34d323e admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.422838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7629e-2f80-4e0c-8fc5-68c9e34d323e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.424741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7629e-2f80-4e0c-8fc5-68c9e34d323e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.447346 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656713-6584-4888-bba2-1a5388877ef7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.448541 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea51ccb-aa50-43e3-8b42-65e5750b6079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.449194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea51ccb-aa50-43e3-8b42-65e5750b6079 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.449482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea51ccb-aa50-43e3-8b42-65e5750b6079 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.449766 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea51ccb-aa50-43e3-8b42-65e5750b6079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.452269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3ea51ccb-aa50-43e3-8b42-65e5750b6079 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:18.511421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea51ccb-aa50-43e3-8b42-65e5750b6079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.513389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea51ccb-aa50-43e3-8b42-65e5750b6079 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.522362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1295/2574] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:36:18.540039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1280/2575] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:18 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:18.541196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea51ccb-aa50-43e3-8b42-65e5750b6079 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.553561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea51ccb-aa50-43e3-8b42-65e5750b6079 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.557677 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2171-1fbb-4264-a934-6aadc09c4788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.558422 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2171-1fbb-4264-a934-6aadc09c4788 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.558747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2171-1fbb-4264-a934-6aadc09c4788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.559034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2171-1fbb-4264-a934-6aadc09c4788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.559924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa2171-1fbb-4264-a934-6aadc09c4788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.561663 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa2171-1fbb-4264-a934-6aadc09c4788 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.572311 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1296/2576] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:18.577314 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7629e-2f80-4e0c-8fc5-68c9e34d323e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.587610 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7629e-2f80-4e0c-8fc5-68c9e34d323e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.593112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af8bc6-d62a-418f-b626-8de125a7f81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.593771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af8bc6-d62a-418f-b626-8de125a7f81e admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.594055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af8bc6-d62a-418f-b626-8de125a7f81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.594330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af8bc6-d62a-418f-b626-8de125a7f81e admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.595094 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af8bc6-d62a-418f-b626-8de125a7f81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.600629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af8bc6-d62a-418f-b626-8de125a7f81e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.600629 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:18.610453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1281/2577] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:18.614578 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2171-1fbb-4264-a934-6aadc09c4788 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.625694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2171-1fbb-4264-a934-6aadc09c4788 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.630951 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57538d-258f-491e-8c32-1b6df027278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.631685 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57538d-258f-491e-8c32-1b6df027278b admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.631973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57538d-258f-491e-8c32-1b6df027278b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.632253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57538d-258f-491e-8c32-1b6df027278b admin admin] PATH_INFO: `/v3/projects/ff5ebee4cf6c47f9adee30f9c8cd6ba1/users/0beb4f30b5c34ab0afdbfbf46dbfede5/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.650020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57538d-258f-491e-8c32-1b6df027278b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff5ebee4cf6c47f9adee30f9c8cd6ba1, user_id=0beb4f30b5c34ab0afdbfbf46dbfede5, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.652730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57538d-258f-491e-8c32-1b6df027278b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.772120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1297/2578] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:18 2022] PUT /identity/v3/projects/ff5ebee4cf6c47f9adee30f9c8cd6ba1/users/0beb4f30b5c34ab0afdbfbf46dbfede5/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:18.777207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a51ff-126d-4307-93e2-d27767064b00 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.779674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a51ff-126d-4307-93e2-d27767064b00 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.779674 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a51ff-126d-4307-93e2-d27767064b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.796142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2f0a51ff-126d-4307-93e2-d27767064b00 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:18.797362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2f0a51ff-126d-4307-93e2-d27767064b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:18.858119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1282/2579] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:18.874083 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57538d-258f-491e-8c32-1b6df027278b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:18.876041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-df57538d-258f-491e-8c32-1b6df027278b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:18.904888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:18.937681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57538d-258f-491e-8c32-1b6df027278b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:18.942174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c03005-b950-4421-b60d-2d256f334fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:18.943121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c03005-b950-4421-b60d-2d256f334fbb service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:18.947477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c03005-b950-4421-b60d-2d256f334fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:18.947477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c03005-b950-4421-b60d-2d256f334fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:18.953525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c03005-b950-4421-b60d-2d256f334fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:18.955007 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c03005-b950-4421-b60d-2d256f334fbb service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:18.965244 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1298/2580] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:24.530369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a51ff-126d-4307-93e2-d27767064b00 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:24.542242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a51ff-126d-4307-93e2-d27767064b00 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:24.548775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:24.550164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:24.550627 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:24.551014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] PATH_INFO: `/v3/users/051eb8ce715d4466a50beee711e0f303` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:24.555658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051eb8ce715d4466a50beee711e0f303)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:24.557636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:24.828467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051eb8ce715d4466a50beee711e0f303'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:24.832428 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051eb8ce715d4466a50beee711e0f303'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:24.851753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051eb8ce715d4466a50beee711e0f303'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:24.860986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051eb8ce715d4466a50beee711e0f303'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:24.871914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac1421-592d-4795-9dd2-61827f06e729 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:24.872471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac1421-592d-4795-9dd2-61827f06e729 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:24.872659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac1421-592d-4795-9dd2-61827f06e729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:24.904043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-29ac1421-592d-4795-9dd2-61827f06e729 None None] MFA Rules not processed for user `0beb4f30b5c34ab0afdbfbf46dbfede5`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:24.906582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-29ac1421-592d-4795-9dd2-61827f06e729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:24.936046 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-8bd44de0-f5b4-4965-b6f7-dd0ab3652a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051eb8ce715d4466a50beee711e0f303'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:24.953835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1283/2581] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:24 2022] DELETE /identity/v3/users/051eb8ce715d4466a50beee711e0f303 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:24.957906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d0f4c-a362-477a-a4dd-2425ac2acfa6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:24.958506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d0f4c-a362-477a-a4dd-2425ac2acfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:24.958794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d0f4c-a362-477a-a4dd-2425ac2acfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:24.978890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fd1d0f4c-a362-477a-a4dd-2425ac2acfa6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:24.980508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fd1d0f4c-a362-477a-a4dd-2425ac2acfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:24.991638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1299/2582] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:24.996212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac1421-592d-4795-9dd2-61827f06e729 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:24.999363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-29ac1421-592d-4795-9dd2-61827f06e729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:25.014979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac1421-592d-4795-9dd2-61827f06e729 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:25.054820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1284/2583] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:25.064449 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d0f4c-a362-477a-a4dd-2425ac2acfa6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:25.065864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fd1d0f4c-a362-477a-a4dd-2425ac2acfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:25.082641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:25.096606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1eaee-ae40-4360-992c-573be102f2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:25.097543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1eaee-ae40-4360-992c-573be102f2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.097543 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1eaee-ae40-4360-992c-573be102f2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.099315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1eaee-ae40-4360-992c-573be102f2d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.099315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1eaee-ae40-4360-992c-573be102f2d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:25.100632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1eaee-ae40-4360-992c-573be102f2d3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:25.139037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d0f4c-a362-477a-a4dd-2425ac2acfa6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:25.142711 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad59d25-5fe7-4511-bda0-234eaa0b10d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:25.143414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad59d25-5fe7-4511-bda0-234eaa0b10d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.143684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad59d25-5fe7-4511-bda0-234eaa0b10d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.143965 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad59d25-5fe7-4511-bda0-234eaa0b10d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.156083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad59d25-5fe7-4511-bda0-234eaa0b10d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:25.157525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad59d25-5fe7-4511-bda0-234eaa0b10d9 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:25.167426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1285/2584] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:36:25.217716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1300/2585] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:24 2022] POST /identity/v3/projects => generated 379 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:25.221712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad59d25-5fe7-4511-bda0-234eaa0b10d9 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:25.231379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad59d25-5fe7-4511-bda0-234eaa0b10d9 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:25.289697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131a08c-eb59-449b-a52a-7fcf6c57b31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:25.290364 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131a08c-eb59-449b-a52a-7fcf6c57b31d admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.290680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131a08c-eb59-449b-a52a-7fcf6c57b31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.290953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131a08c-eb59-449b-a52a-7fcf6c57b31d admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.291639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6131a08c-eb59-449b-a52a-7fcf6c57b31d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:25.293540 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6131a08c-eb59-449b-a52a-7fcf6c57b31d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:25.392540 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1eaee-ae40-4360-992c-573be102f2d3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:25.396690 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d4c1eaee-ae40-4360-992c-573be102f2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:25.411358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1286/2586] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:25 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:25.413061 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131a08c-eb59-449b-a52a-7fcf6c57b31d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:25.419113 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1eaee-ae40-4360-992c-573be102f2d3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:25.427371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131a08c-eb59-449b-a52a-7fcf6c57b31d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:25.427665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:25.429589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.430679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.431551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] PATH_INFO: `/v3/users/05cef7fc49ad4901851d9888cbbc91fa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.433379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8c913-eec7-4c25-b10a-71be52821fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:25.433628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05cef7fc49ad4901851d9888cbbc91fa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:25.434495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8c913-eec7-4c25-b10a-71be52821fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.434864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8c913-eec7-4c25-b10a-71be52821fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.435212 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8c913-eec7-4c25-b10a-71be52821fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.436344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8c913-eec7-4c25-b10a-71be52821fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:25.437228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:25.438097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8c913-eec7-4c25-b10a-71be52821fbc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:25.451118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1287/2587] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:25.459451 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8c913-eec7-4c25-b10a-71be52821fbc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:25.476510 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8c913-eec7-4c25-b10a-71be52821fbc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:25.485107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753701d-2877-48df-a585-998fa21e26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:25.487578 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753701d-2877-48df-a585-998fa21e26ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.487578 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753701d-2877-48df-a585-998fa21e26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.487578 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753701d-2877-48df-a585-998fa21e26ac admin admin] PATH_INFO: `/v3/projects/b46af4302be94737a202b2f1d3664547/users/09388632784a43eebbfb94bd3c0f29f6/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.508886 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753701d-2877-48df-a585-998fa21e26ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46af4302be94737a202b2f1d3664547, user_id=09388632784a43eebbfb94bd3c0f29f6, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:25.512342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753701d-2877-48df-a585-998fa21e26ac admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:25.621730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1288/2588] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:25 2022] PUT /identity/v3/projects/b46af4302be94737a202b2f1d3664547/users/09388632784a43eebbfb94bd3c0f29f6/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:25.626254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753701d-2877-48df-a585-998fa21e26ac admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:25.654773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:25.674171 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05cef7fc49ad4901851d9888cbbc91fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:25.698089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05cef7fc49ad4901851d9888cbbc91fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:25.698981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05cef7fc49ad4901851d9888cbbc91fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:25.701352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05cef7fc49ad4901851d9888cbbc91fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:25.741034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753701d-2877-48df-a585-998fa21e26ac admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:25.747605 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f10a02-839b-4c7a-84a5-3bce137599c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:25.749063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f10a02-839b-4c7a-84a5-3bce137599c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.749422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f10a02-839b-4c7a-84a5-3bce137599c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.749772 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f10a02-839b-4c7a-84a5-3bce137599c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.750807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f10a02-839b-4c7a-84a5-3bce137599c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:25.752467 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f10a02-839b-4c7a-84a5-3bce137599c4 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:25.765755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1289/2589] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:25.766075 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f10a02-839b-4c7a-84a5-3bce137599c4 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:25.768243 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b8f10a02-839b-4c7a-84a5-3bce137599c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:25.776123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05cef7fc49ad4901851d9888cbbc91fa'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:25.777393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:25.784585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f10a02-839b-4c7a-84a5-3bce137599c4 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:25.790738 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:25.791702 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.791909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.792188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] PATH_INFO: `/v3/projects/ac5fcad06279414e868008a595e83fe2` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.793982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac5fcad06279414e868008a595e83fe2)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:25.796890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:25.813741 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1301/2590] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:25 2022] DELETE /identity/v3/users/05cef7fc49ad4901851d9888cbbc91fa => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:25.814696 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:25.816096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:25.885957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680340c9-6c7d-4b7b-b6de-d8a27989fa89 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:25.891704 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351711ed-21cf-4e51-b010-ddcdeb54d8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:25.892654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351711ed-21cf-4e51-b010-ddcdeb54d8ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.893020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351711ed-21cf-4e51-b010-ddcdeb54d8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.893294 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351711ed-21cf-4e51-b010-ddcdeb54d8ad admin admin] PATH_INFO: `/v3/projects/b46af4302be94737a202b2f1d3664547/users/09388632784a43eebbfb94bd3c0f29f6/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.895410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351711ed-21cf-4e51-b010-ddcdeb54d8ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46af4302be94737a202b2f1d3664547, user_id=09388632784a43eebbfb94bd3c0f29f6, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:25.898053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351711ed-21cf-4e51-b010-ddcdeb54d8ad admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:25.911053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1302/2591] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:25 2022] PUT /identity/v3/projects/b46af4302be94737a202b2f1d3664547/users/09388632784a43eebbfb94bd3c0f29f6/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:25.912319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149722a9-458c-4c7d-b114-9c57f53e1c15 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:25.912606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149722a9-458c-4c7d-b114-9c57f53e1c15 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:25.912884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149722a9-458c-4c7d-b114-9c57f53e1c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:25.942959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-149722a9-458c-4c7d-b114-9c57f53e1c15 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:25.944240 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-149722a9-458c-4c7d-b114-9c57f53e1c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:25.955512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac5fcad06279414e868008a595e83fe2', 'request_id': 'req-d9a3a064-beca-4e29-9869-d98c83b9fbbd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:25.961446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac5fcad06279414e868008a595e83fe2', 'request_id': 'req-d9a3a064-beca-4e29-9869-d98c83b9fbbd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:26.021223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac5fcad06279414e868008a595e83fe2', 'request_id': 'req-d9a3a064-beca-4e29-9869-d98c83b9fbbd'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:26.038616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1303/2592] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:26.041688 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] The token cache is being invalidate because project ac5fcad06279414e868008a595e83fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:26.042321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:26.043351 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:26.045295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1290/2593] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:25 2022] DELETE /identity/v3/projects/ac5fcad06279414e868008a595e83fe2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:26.048757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149722a9-458c-4c7d-b114-9c57f53e1c15 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:26.049425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-149722a9-458c-4c7d-b114-9c57f53e1c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:26.050662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:26.052589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:26.073430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3a064-beca-4e29-9869-d98c83b9fbbd admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:26.079905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:26.081365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:26.081914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:26.082474 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] PATH_INFO: `/v3/users/8dc4b1816c2f45b8bf74b5486648deab` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:26.084573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dc4b1816c2f45b8bf74b5486648deab)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:26.087122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:26.131073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:26.137986 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149722a9-458c-4c7d-b114-9c57f53e1c15 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:26.142773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:26.143997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:26.143997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:26.143997 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:26.146028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:26.161389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:26.163227 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:26.175478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1304/2594] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:26.354422 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dc4b1816c2f45b8bf74b5486648deab'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:26.355171 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dc4b1816c2f45b8bf74b5486648deab'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:26.382609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dc4b1816c2f45b8bf74b5486648deab'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:26.382609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dc4b1816c2f45b8bf74b5486648deab'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:26.473383 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dc4b1816c2f45b8bf74b5486648deab'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:26.489873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1291/2595] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:26 2022] DELETE /identity/v3/users/8dc4b1816c2f45b8bf74b5486648deab => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:26.994832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:26.996768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:27.015933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795169a-77bb-4713-ad9e-a299ccf4afd1 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:27.132778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:27.133732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.134012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.134354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] PATH_INFO: `/v3/projects/0558cab0498a4ccb830fe7ad72066401` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.135611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0558cab0498a4ccb830fe7ad72066401)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:27.137638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:27.177124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:27.180933 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:27.218140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f732c6-f3f6-4d87-bdbe-b18bf10a4511 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:27.225899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:27.227700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.228566 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.229059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] PATH_INFO: `/v3/projects/befc5073e6094c0a8ca911cf5e0dff00` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.241914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=befc5073e6094c0a8ca911cf5e0dff00)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:27.244109 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:27.305942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0558cab0498a4ccb830fe7ad72066401', 'request_id': 'req-1c792ad0-f35a-4af1-bc32-376a47f2276f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.314429 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0558cab0498a4ccb830fe7ad72066401', 'request_id': 'req-1c792ad0-f35a-4af1-bc32-376a47f2276f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.399605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0558cab0498a4ccb830fe7ad72066401', 'request_id': 'req-1c792ad0-f35a-4af1-bc32-376a47f2276f'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.417379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] The token cache is being invalidate because project 0558cab0498a4ccb830fe7ad72066401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:27.417832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.418620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-1c792ad0-f35a-4af1-bc32-376a47f2276f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.430155 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'befc5073e6094c0a8ca911cf5e0dff00', 'request_id': 'req-511c1f2f-3ad0-4096-968e-9f53b91108b7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.431177 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1305/2596] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:26 2022] DELETE /identity/v3/projects/0558cab0498a4ccb830fe7ad72066401 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:27.435915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c99b6c-3012-492d-99dc-e2ce5ca479b3 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.436134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=76,keystone:DELETE=22,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:27.436787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c99b6c-3012-492d-99dc-e2ce5ca479b3 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.437174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c99b6c-3012-492d-99dc-e2ce5ca479b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.442693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'befc5073e6094c0a8ca911cf5e0dff00', 'request_id': 'req-511c1f2f-3ad0-4096-968e-9f53b91108b7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.489838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-75c99b6c-3012-492d-99dc-e2ce5ca479b3 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:27.493523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-75c99b6c-3012-492d-99dc-e2ce5ca479b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:27.576973 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'befc5073e6094c0a8ca911cf5e0dff00', 'request_id': 'req-511c1f2f-3ad0-4096-968e-9f53b91108b7'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.593226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] The token cache is being invalidate because project befc5073e6094c0a8ca911cf5e0dff00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:27.593586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.594237 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-511c1f2f-3ad0-4096-968e-9f53b91108b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:27.598485 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1292/2597] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:27 2022] DELETE /identity/v3/projects/befc5073e6094c0a8ca911cf5e0dff00 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:36:27.602509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e3118-ca9b-40ae-9185-57795ba687b6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.602828 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e3118-ca9b-40ae-9185-57795ba687b6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.603114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e3118-ca9b-40ae-9185-57795ba687b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.610512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1306/2598] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:27.621035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335c20b-48ae-4812-b13b-82601cdc2471 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.621035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335c20b-48ae-4812-b13b-82601cdc2471 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.621035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335c20b-48ae-4812-b13b-82601cdc2471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.626885 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d38e3118-ca9b-40ae-9185-57795ba687b6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:27.628032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d38e3118-ca9b-40ae-9185-57795ba687b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:27.640643 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1293/2599] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:27.645462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c335c20b-48ae-4812-b13b-82601cdc2471 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:27.649452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c335c20b-48ae-4812-b13b-82601cdc2471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:27.649583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6ce9b-7922-4a97-a7c5-4cc37b66c4c7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.650169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6ce9b-7922-4a97-a7c5-4cc37b66c4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.651359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6ce9b-7922-4a97-a7c5-4cc37b66c4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.656362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1307/2600] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:27.660421 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335c20b-48ae-4812-b13b-82601cdc2471 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:27.670445 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335c20b-48ae-4812-b13b-82601cdc2471 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:27.676632 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4fd7a-fcba-44ae-a550-d427593821b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:27.677607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4fd7a-fcba-44ae-a550-d427593821b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.678060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4fd7a-fcba-44ae-a550-d427593821b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.678345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4fd7a-fcba-44ae-a550-d427593821b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.683744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4fd7a-fcba-44ae-a550-d427593821b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:36:27.683744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4fd7a-fcba-44ae-a550-d427593821b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:27.683744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4fd7a-fcba-44ae-a550-d427593821b0 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:27.684022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-17a6ce9b-7922-4a97-a7c5-4cc37b66c4c7 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:27.684022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-17a6ce9b-7922-4a97-a7c5-4cc37b66c4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:27.696122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1308/2601] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:36:27.696389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1294/2602] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:27.700470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4fd7a-fcba-44ae-a550-d427593821b0 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:27.705631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6ce9b-7922-4a97-a7c5-4cc37b66c4c7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:27.712554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:27.714081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4fd7a-fcba-44ae-a550-d427593821b0 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:27.714846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6ce9b-7922-4a97-a7c5-4cc37b66c4c7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:27.719384 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1cdc9-1777-40ef-9cda-1735be417d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:27.719789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f85af-77c1-4b61-8739-6c168a0312d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:27.720041 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1cdc9-1777-40ef-9cda-1735be417d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.720324 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1cdc9-1777-40ef-9cda-1735be417d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.720600 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1cdc9-1777-40ef-9cda-1735be417d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.720842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f85af-77c1-4b61-8739-6c168a0312d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.721220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f85af-77c1-4b61-8739-6c168a0312d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.721326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1cdc9-1777-40ef-9cda-1735be417d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:27.721667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f85af-77c1-4b61-8739-6c168a0312d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.722530 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f85af-77c1-4b61-8739-6c168a0312d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:36:27.722947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f85af-77c1-4b61-8739-6c168a0312d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:27.723310 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1cdc9-1777-40ef-9cda-1735be417d2e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:27.724747 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f85af-77c1-4b61-8739-6c168a0312d7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:27.742969 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1295/2603] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:36:27.744074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f85af-77c1-4b61-8739-6c168a0312d7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:27.755620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f85af-77c1-4b61-8739-6c168a0312d7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:27.760669 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e41e16-552e-4495-a45a-e3e5e08ca86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:27.761321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e41e16-552e-4495-a45a-e3e5e08ca86f admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.761602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e41e16-552e-4495-a45a-e3e5e08ca86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.761873 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e41e16-552e-4495-a45a-e3e5e08ca86f admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:27.762635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e41e16-552e-4495-a45a-e3e5e08ca86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:27.765511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e41e16-552e-4495-a45a-e3e5e08ca86f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:27.877772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1309/2604] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:27 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:27.879085 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1cdc9-1777-40ef-9cda-1735be417d2e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:27.916443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1296/2605] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:27 2022] POST /identity/v3/projects => generated 401 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:27.918082 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e41e16-552e-4495-a45a-e3e5e08ca86f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:27.980313 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1cdc9-1777-40ef-9cda-1735be417d2e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:27.991597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e41e16-552e-4495-a45a-e3e5e08ca86f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:27.996908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33871a9-a0b9-44d9-903a-4e094a6705fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:27.997821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33871a9-a0b9-44d9-903a-4e094a6705fc admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:27.998311 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33871a9-a0b9-44d9-903a-4e094a6705fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:27.999076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33871a9-a0b9-44d9-903a-4e094a6705fc admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.000116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33871a9-a0b9-44d9-903a-4e094a6705fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.002274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33871a9-a0b9-44d9-903a-4e094a6705fc admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.057684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c58d8-e687-4504-a290-39d93e6f079d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:28.057684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c58d8-e687-4504-a290-39d93e6f079d admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.057684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c58d8-e687-4504-a290-39d93e6f079d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.057684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c58d8-e687-4504-a290-39d93e6f079d admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.058305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c58d8-e687-4504-a290-39d93e6f079d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.058305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c58d8-e687-4504-a290-39d93e6f079d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.097347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1297/2606] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:27 2022] POST /identity/v3/users => generated 301 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:28.101342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33871a9-a0b9-44d9-903a-4e094a6705fc admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.113666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33871a9-a0b9-44d9-903a-4e094a6705fc admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:28.119051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711827a-a179-4911-8f6b-1e99dd2906ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:28.119891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711827a-a179-4911-8f6b-1e99dd2906ea admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.120559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711827a-a179-4911-8f6b-1e99dd2906ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.120559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711827a-a179-4911-8f6b-1e99dd2906ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.121559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711827a-a179-4911-8f6b-1e99dd2906ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.123984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711827a-a179-4911-8f6b-1e99dd2906ea admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.139750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1310/2607] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:27 2022] POST /identity/v3/users => generated 287 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:28.143008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c58d8-e687-4504-a290-39d93e6f079d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.155032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1298/2608] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:28.157376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711827a-a179-4911-8f6b-1e99dd2906ea admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.191329 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711827a-a179-4911-8f6b-1e99dd2906ea admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:28.193919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c58d8-e687-4504-a290-39d93e6f079d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:28.197158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bc654-19e8-4076-93d4-02cba74e1d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:28.198025 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bc654-19e8-4076-93d4-02cba74e1d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.198442 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bc654-19e8-4076-93d4-02cba74e1d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.198804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bc654-19e8-4076-93d4-02cba74e1d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.199542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bc654-19e8-4076-93d4-02cba74e1d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.199664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052dc28f-7dac-4898-8f18-60cfc02f3606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:28.200476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052dc28f-7dac-4898-8f18-60cfc02f3606 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.200777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052dc28f-7dac-4898-8f18-60cfc02f3606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.201260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052dc28f-7dac-4898-8f18-60cfc02f3606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.201602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bc654-19e8-4076-93d4-02cba74e1d9b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.202218 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052dc28f-7dac-4898-8f18-60cfc02f3606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.204096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052dc28f-7dac-4898-8f18-60cfc02f3606 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.217931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1299/2610] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:28.218142 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1311/2610] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:28.221253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052dc28f-7dac-4898-8f18-60cfc02f3606 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.222371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bc654-19e8-4076-93d4-02cba74e1d9b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.235515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bc654-19e8-4076-93d4-02cba74e1d9b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:28.240732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052dc28f-7dac-4898-8f18-60cfc02f3606 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:28.241521 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb8dcd-c877-49b8-a27b-4e477160e45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:28.242327 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb8dcd-c877-49b8-a27b-4e477160e45f admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.242653 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb8dcd-c877-49b8-a27b-4e477160e45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.243031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb8dcd-c877-49b8-a27b-4e477160e45f admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.243882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb8dcd-c877-49b8-a27b-4e477160e45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.245587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb8dcd-c877-49b8-a27b-4e477160e45f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.248055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2bf2c-3a04-408b-ad9d-254040e4d32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:28.249474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2bf2c-3a04-408b-ad9d-254040e4d32a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.262190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2bf2c-3a04-408b-ad9d-254040e4d32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.262190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2bf2c-3a04-408b-ad9d-254040e4d32a admin admin] PATH_INFO: `/v3/projects/43aff8e7c37542eb9aac50045674d1bc/users/fa208dd76b074319b49f42a2a64af728/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.267842 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1300/2611] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:28.268787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb8dcd-c877-49b8-a27b-4e477160e45f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.278963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2bf2c-3a04-408b-ad9d-254040e4d32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=43aff8e7c37542eb9aac50045674d1bc, user_id=fa208dd76b074319b49f42a2a64af728, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.279741 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb8dcd-c877-49b8-a27b-4e477160e45f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:28.282728 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2bf2c-3a04-408b-ad9d-254040e4d32a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.285795 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17fdce-d310-4185-8739-2edf71af5db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:28.288168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17fdce-d310-4185-8739-2edf71af5db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.288512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17fdce-d310-4185-8739-2edf71af5db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.288800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17fdce-d310-4185-8739-2edf71af5db0 admin admin] PATH_INFO: `/v3/projects/db36e8d7a4164e4d9f346ec70d2bf5e6/users/47501d80d8db42e78ef2ab5cc7bf56bc/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.307503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17fdce-d310-4185-8739-2edf71af5db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db36e8d7a4164e4d9f346ec70d2bf5e6, user_id=47501d80d8db42e78ef2ab5cc7bf56bc, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.310569 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17fdce-d310-4185-8739-2edf71af5db0 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.360622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1312/2612] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:28 2022] PUT /identity/v3/projects/43aff8e7c37542eb9aac50045674d1bc/users/fa208dd76b074319b49f42a2a64af728/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:28.370945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27743e0-63c2-460c-adbf-bd60245e7658 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.371444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27743e0-63c2-460c-adbf-bd60245e7658 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.371736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27743e0-63c2-460c-adbf-bd60245e7658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.386377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1301/2613] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:28 2022] PUT /identity/v3/projects/db36e8d7a4164e4d9f346ec70d2bf5e6/users/47501d80d8db42e78ef2ab5cc7bf56bc/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:28.399128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a7a9c-1f17-4b5e-a5d1-f2cba24dcfe5 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.399582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a7a9c-1f17-4b5e-a5d1-f2cba24dcfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.399958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a7a9c-1f17-4b5e-a5d1-f2cba24dcfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.405814 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d27743e0-63c2-460c-adbf-bd60245e7658 None None] MFA Rules not processed for user `fa208dd76b074319b49f42a2a64af728`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:28.413023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d27743e0-63c2-460c-adbf-bd60245e7658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:28.437695 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-891a7a9c-1f17-4b5e-a5d1-f2cba24dcfe5 None None] MFA Rules not processed for user `47501d80d8db42e78ef2ab5cc7bf56bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:28.439432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-891a7a9c-1f17-4b5e-a5d1-f2cba24dcfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:28.506808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1313/2614] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:28.512075 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27743e0-63c2-460c-adbf-bd60245e7658 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.537289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1302/2615] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:28.538684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a7a9c-1f17-4b5e-a5d1-f2cba24dcfe5 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.615024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27743e0-63c2-460c-adbf-bd60245e7658 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:28.626080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27efd6a-7ff9-449e-95f2-0c9f81ff1662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:28.627527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27efd6a-7ff9-449e-95f2-0c9f81ff1662 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.628386 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27efd6a-7ff9-449e-95f2-0c9f81ff1662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.628923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27efd6a-7ff9-449e-95f2-0c9f81ff1662 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.629764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27efd6a-7ff9-449e-95f2-0c9f81ff1662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.632256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27efd6a-7ff9-449e-95f2-0c9f81ff1662 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.654931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a7a9c-1f17-4b5e-a5d1-f2cba24dcfe5 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:28.654931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e366aa-abc3-45e7-9553-96c54857fc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:28.654931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e366aa-abc3-45e7-9553-96c54857fc02 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:28.654931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e366aa-abc3-45e7-9553-96c54857fc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:28.655472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e366aa-abc3-45e7-9553-96c54857fc02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:28.655472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e366aa-abc3-45e7-9553-96c54857fc02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:28.657446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e366aa-abc3-45e7-9553-96c54857fc02 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:28.784418 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1314/2616] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:28 2022] POST /identity/v3/projects => generated 399 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:28.790874 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27efd6a-7ff9-449e-95f2-0c9f81ff1662 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.834718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1303/2617] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:28 2022] POST /identity/v3/projects => generated 373 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:28.836455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e366aa-abc3-45e7-9553-96c54857fc02 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:28.899913 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27efd6a-7ff9-449e-95f2-0c9f81ff1662 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.035711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6726d8-bf4d-4f1b-b35d-5642ebfe87f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.036547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6726d8-bf4d-4f1b-b35d-5642ebfe87f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.041070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6726d8-bf4d-4f1b-b35d-5642ebfe87f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.041070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6726d8-bf4d-4f1b-b35d-5642ebfe87f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.041070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6726d8-bf4d-4f1b-b35d-5642ebfe87f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.041070 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6726d8-bf4d-4f1b-b35d-5642ebfe87f9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.064520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e366aa-abc3-45e7-9553-96c54857fc02 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.078238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55d31c-b73a-4aaa-8eae-40fb1fc97a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.079391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55d31c-b73a-4aaa-8eae-40fb1fc97a44 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.079860 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55d31c-b73a-4aaa-8eae-40fb1fc97a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.080323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55d31c-b73a-4aaa-8eae-40fb1fc97a44 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.081560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55d31c-b73a-4aaa-8eae-40fb1fc97a44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.085345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55d31c-b73a-4aaa-8eae-40fb1fc97a44 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.149305 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1315/2618] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:28 2022] POST /identity/v3/users => generated 306 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:29.153984 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6726d8-bf4d-4f1b-b35d-5642ebfe87f9 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.168439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6726d8-bf4d-4f1b-b35d-5642ebfe87f9 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.177706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f312b4-b815-476f-af2a-c21f6c1da965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.178659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f312b4-b815-476f-af2a-c21f6c1da965 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.179302 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f312b4-b815-476f-af2a-c21f6c1da965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.181170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f312b4-b815-476f-af2a-c21f6c1da965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.182557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f312b4-b815-476f-af2a-c21f6c1da965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.184358 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f312b4-b815-476f-af2a-c21f6c1da965 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.198339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1304/2619] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:28 2022] POST /identity/v3/users => generated 293 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:29.199770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55d31c-b73a-4aaa-8eae-40fb1fc97a44 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.211707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55d31c-b73a-4aaa-8eae-40fb1fc97a44 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.219895 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1316/2620] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:29.222860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f312b4-b815-476f-af2a-c21f6c1da965 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.223229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=2,keystone:INSERT=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:29.225731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9226f-d9c3-45bd-98c3-13cf00e0af88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.227208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9226f-d9c3-45bd-98c3-13cf00e0af88 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.227343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9226f-d9c3-45bd-98c3-13cf00e0af88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.227684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9226f-d9c3-45bd-98c3-13cf00e0af88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.228499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9226f-d9c3-45bd-98c3-13cf00e0af88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.230694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9226f-d9c3-45bd-98c3-13cf00e0af88 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.242093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f312b4-b815-476f-af2a-c21f6c1da965 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.243821 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1305/2621] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:29.247657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9226f-d9c3-45bd-98c3-13cf00e0af88 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.251487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86211e-9491-4e01-b9bb-0eb98b7db955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.252703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86211e-9491-4e01-b9bb-0eb98b7db955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.252992 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86211e-9491-4e01-b9bb-0eb98b7db955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.253274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86211e-9491-4e01-b9bb-0eb98b7db955 admin admin] PATH_INFO: `/v3/projects/84953e645e1e48a0831e2244cbeb24ff/users/43d6d2e404c74f15b6ecdf61331f904d/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.260337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9226f-d9c3-45bd-98c3-13cf00e0af88 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.266319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdfa92-1963-4a8c-8ad4-0bf5400e33a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.273288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdfa92-1963-4a8c-8ad4-0bf5400e33a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.273288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdfa92-1963-4a8c-8ad4-0bf5400e33a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.273288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdfa92-1963-4a8c-8ad4-0bf5400e33a1 admin admin] PATH_INFO: `/v3/projects/0b2e0774415b4d64a86ec47f2a7b936f/users/e0dcb6cef8774edc8bd257fbabcc8596/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.288263 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdfa92-1963-4a8c-8ad4-0bf5400e33a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2e0774415b4d64a86ec47f2a7b936f, user_id=e0dcb6cef8774edc8bd257fbabcc8596, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.292021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86211e-9491-4e01-b9bb-0eb98b7db955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84953e645e1e48a0831e2244cbeb24ff, user_id=43d6d2e404c74f15b6ecdf61331f904d, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.295631 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86211e-9491-4e01-b9bb-0eb98b7db955 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.297826 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbdfa92-1963-4a8c-8ad4-0bf5400e33a1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.382626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1317/2622] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:29 2022] PUT /identity/v3/projects/84953e645e1e48a0831e2244cbeb24ff/users/43d6d2e404c74f15b6ecdf61331f904d/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:29.394038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86211e-9491-4e01-b9bb-0eb98b7db955 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.395825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1306/2623] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:29 2022] PUT /identity/v3/projects/0b2e0774415b4d64a86ec47f2a7b936f/users/e0dcb6cef8774edc8bd257fbabcc8596/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:29.400101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdfa92-1963-4a8c-8ad4-0bf5400e33a1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.466939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:29.506238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86211e-9491-4e01-b9bb-0eb98b7db955 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.506770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbdfa92-1963-4a8c-8ad4-0bf5400e33a1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.521118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c5cb3-4506-4e63-973a-da7963bda697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.521819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c5cb3-4506-4e63-973a-da7963bda697 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.522637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c5cb3-4506-4e63-973a-da7963bda697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.526021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c5cb3-4506-4e63-973a-da7963bda697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.534811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d11803-e77e-4cd7-ba92-b17c09a2d152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.534811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d11803-e77e-4cd7-ba92-b17c09a2d152 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.534811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d11803-e77e-4cd7-ba92-b17c09a2d152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.534811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d11803-e77e-4cd7-ba92-b17c09a2d152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.535141 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d11803-e77e-4cd7-ba92-b17c09a2d152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.535215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c5cb3-4506-4e63-973a-da7963bda697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.535215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c5cb3-4506-4e63-973a-da7963bda697 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.535370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d11803-e77e-4cd7-ba92-b17c09a2d152 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.568326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1307/2624] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:29.577128 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1318/2625] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:29.577857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d11803-e77e-4cd7-ba92-b17c09a2d152 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.581401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c5cb3-4506-4e63-973a-da7963bda697 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.597948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d11803-e77e-4cd7-ba92-b17c09a2d152 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.614190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c5cb3-4506-4e63-973a-da7963bda697 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.615528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb19905-b68b-429d-8d03-73be69876851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.616568 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb19905-b68b-429d-8d03-73be69876851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.617018 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb19905-b68b-429d-8d03-73be69876851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.617361 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb19905-b68b-429d-8d03-73be69876851 admin admin] PATH_INFO: `/v3/projects/0b2e0774415b4d64a86ec47f2a7b936f/users/e0dcb6cef8774edc8bd257fbabcc8596/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.619641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68539709-8b19-407a-8245-e9540fdbfea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.619836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb19905-b68b-429d-8d03-73be69876851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2e0774415b4d64a86ec47f2a7b936f, user_id=e0dcb6cef8774edc8bd257fbabcc8596, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.621777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539709-8b19-407a-8245-e9540fdbfea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.621777 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539709-8b19-407a-8245-e9540fdbfea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.622018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539709-8b19-407a-8245-e9540fdbfea8 admin admin] PATH_INFO: `/v3/projects/84953e645e1e48a0831e2244cbeb24ff/users/43d6d2e404c74f15b6ecdf61331f904d/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.623602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb19905-b68b-429d-8d03-73be69876851 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.624471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68539709-8b19-407a-8245-e9540fdbfea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84953e645e1e48a0831e2244cbeb24ff, user_id=43d6d2e404c74f15b6ecdf61331f904d, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.627899 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68539709-8b19-407a-8245-e9540fdbfea8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.644616 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1308/2626] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:29 2022] PUT /identity/v3/projects/0b2e0774415b4d64a86ec47f2a7b936f/users/e0dcb6cef8774edc8bd257fbabcc8596/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:29.650552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d9f15-5a6e-4788-a8ec-7041ea05eb3b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.651491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d9f15-5a6e-4788-a8ec-7041ea05eb3b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.651491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d9f15-5a6e-4788-a8ec-7041ea05eb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.673318 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1319/2627] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:29 2022] PUT /identity/v3/projects/84953e645e1e48a0831e2244cbeb24ff/users/43d6d2e404c74f15b6ecdf61331f904d/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:29.675071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a4f6f-13e4-4383-97f1-83ef25d5a06d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.675439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a4f6f-13e4-4383-97f1-83ef25d5a06d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.675793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a4f6f-13e4-4383-97f1-83ef25d5a06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.710240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-181d9f15-5a6e-4788-a8ec-7041ea05eb3b None None] MFA Rules not processed for user `e0dcb6cef8774edc8bd257fbabcc8596`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:29.712835 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-181d9f15-5a6e-4788-a8ec-7041ea05eb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:29.717428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-fd6a4f6f-13e4-4383-97f1-83ef25d5a06d None None] MFA Rules not processed for user `43d6d2e404c74f15b6ecdf61331f904d`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:29.723387 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fd6a4f6f-13e4-4383-97f1-83ef25d5a06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:29.817017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:29.844909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1320/2628] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:29.861097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a4f6f-13e4-4383-97f1-83ef25d5a06d None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.862906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fd6a4f6f-13e4-4383-97f1-83ef25d5a06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:29.881417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1309/2629] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:29.895984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d9f15-5a6e-4788-a8ec-7041ea05eb3b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:29.897660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-181d9f15-5a6e-4788-a8ec-7041ea05eb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:29.953388 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a4f6f-13e4-4383-97f1-83ef25d5a06d None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.957462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464da854-3b2b-46a9-9bcf-fc194831631c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.958233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464da854-3b2b-46a9-9bcf-fc194831631c service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.958606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464da854-3b2b-46a9-9bcf-fc194831631c service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.963037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464da854-3b2b-46a9-9bcf-fc194831631c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.978121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464da854-3b2b-46a9-9bcf-fc194831631c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:29.981274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464da854-3b2b-46a9-9bcf-fc194831631c service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:29.981683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d9f15-5a6e-4788-a8ec-7041ea05eb3b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:29.986593 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7442e-d700-4d0b-9b00-020d2b9e2598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:29.987384 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7442e-d700-4d0b-9b00-020d2b9e2598 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:29.987787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7442e-d700-4d0b-9b00-020d2b9e2598 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:29.988143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7442e-d700-4d0b-9b00-020d2b9e2598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:29.995436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:30.006805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1321/2630] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:36:30.072641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7442e-d700-4d0b-9b00-020d2b9e2598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:30.074119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7442e-d700-4d0b-9b00-020d2b9e2598 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:30.097898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1310/2631] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:36:30.172216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:30.222618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:30.312943 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464da854-3b2b-46a9-9bcf-fc194831631c service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:30.323193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464da854-3b2b-46a9-9bcf-fc194831631c service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:30.327865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e78d42-65bc-49d2-baea-a6f3a0adf298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:30.328570 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e78d42-65bc-49d2-baea-a6f3a0adf298 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:30.328853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e78d42-65bc-49d2-baea-a6f3a0adf298 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:30.329296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e78d42-65bc-49d2-baea-a6f3a0adf298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:30.396269 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e78d42-65bc-49d2-baea-a6f3a0adf298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:30.397944 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e78d42-65bc-49d2-baea-a6f3a0adf298 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:30.410101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1322/2632] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:32.778585 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a56fb8-e4fa-404f-8ab3-f5e77eb38b81 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:32.779760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a56fb8-e4fa-404f-8ab3-f5e77eb38b81 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:32.779760 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a56fb8-e4fa-404f-8ab3-f5e77eb38b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:32.808881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b3a56fb8-e4fa-404f-8ab3-f5e77eb38b81 None None] MFA Rules not processed for user `09388632784a43eebbfb94bd3c0f29f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:32.810108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b3a56fb8-e4fa-404f-8ab3-f5e77eb38b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:32.891127 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1311/2633] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:32.911343 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e78d42-65bc-49d2-baea-a6f3a0adf298 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:32.912795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b1e78d42-65bc-49d2-baea-a6f3a0adf298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:32.996188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e78d42-65bc-49d2-baea-a6f3a0adf298 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:33.003144 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56b5fa-4139-48c0-b0a5-144e8c521cbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:33.003284 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b5fa-4139-48c0-b0a5-144e8c521cbd service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:33.003680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b5fa-4139-48c0-b0a5-144e8c521cbd service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:33.003958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b5fa-4139-48c0-b0a5-144e8c521cbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:33.005815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-da56b5fa-4139-48c0-b0a5-144e8c521cbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:33.069547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56b5fa-4139-48c0-b0a5-144e8c521cbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:33.071537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56b5fa-4139-48c0-b0a5-144e8c521cbd service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:33.083973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1323/2634] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:33.650620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a56fb8-e4fa-404f-8ab3-f5e77eb38b81 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:33.755583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a56fb8-e4fa-404f-8ab3-f5e77eb38b81 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:33.761599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:33.762533 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:33.762814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:33.763098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] PATH_INFO: `/v3/users/47501d80d8db42e78ef2ab5cc7bf56bc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:33.765497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47501d80d8db42e78ef2ab5cc7bf56bc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:33.767522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:34.000012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:34.002556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47501d80d8db42e78ef2ab5cc7bf56bc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:34.003245 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47501d80d8db42e78ef2ab5cc7bf56bc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:34.030531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47501d80d8db42e78ef2ab5cc7bf56bc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:34.033787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47501d80d8db42e78ef2ab5cc7bf56bc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:34.118304 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47501d80d8db42e78ef2ab5cc7bf56bc'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:34.132576 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1312/2635] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:33 2022] DELETE /identity/v3/users/47501d80d8db42e78ef2ab5cc7bf56bc => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:34.136948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482dac-8a03-4b88-8ccd-ac1b8fd4eded None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:34.137416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482dac-8a03-4b88-8ccd-ac1b8fd4eded None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:34.137513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f482dac-8a03-4b88-8ccd-ac1b8fd4eded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:34.144907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:34.158630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9f482dac-8a03-4b88-8ccd-ac1b8fd4eded None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:34.160118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9f482dac-8a03-4b88-8ccd-ac1b8fd4eded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:34.233864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1324/2636] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:34.244402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:34.249976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:34.352976 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea0307-7ddd-48e3-bb75-9a23f945ee79 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:34.360037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8c512-7fe5-4a26-9147-5adf3f400425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:34.360705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8c512-7fe5-4a26-9147-5adf3f400425 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:34.360831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8c512-7fe5-4a26-9147-5adf3f400425 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:34.361217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8c512-7fe5-4a26-9147-5adf3f400425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:34.373472 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8c512-7fe5-4a26-9147-5adf3f400425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:34.375140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab8c512-7fe5-4a26-9147-5adf3f400425 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:34.390967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1313/2637] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:34.912482 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f482dac-8a03-4b88-8ccd-ac1b8fd4eded None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:34.914737 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9f482dac-8a03-4b88-8ccd-ac1b8fd4eded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:34.936389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f482dac-8a03-4b88-8ccd-ac1b8fd4eded None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:34.944658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:34.946523 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:34.947328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:34.947754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] PATH_INFO: `/v3/projects/db36e8d7a4164e4d9f346ec70d2bf5e6` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:34.949738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=db36e8d7a4164e4d9f346ec70d2bf5e6)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:34.953751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:35.137010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db36e8d7a4164e4d9f346ec70d2bf5e6', 'request_id': 'req-5ba8dbee-bc86-4b94-8819-0adf48800fbc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.142843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db36e8d7a4164e4d9f346ec70d2bf5e6', 'request_id': 'req-5ba8dbee-bc86-4b94-8819-0adf48800fbc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.260331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db36e8d7a4164e4d9f346ec70d2bf5e6', 'request_id': 'req-5ba8dbee-bc86-4b94-8819-0adf48800fbc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.288695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] The token cache is being invalidate because project db36e8d7a4164e4d9f346ec70d2bf5e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:35.289123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.292819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.294510 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1325/2638] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:34 2022] DELETE /identity/v3/projects/db36e8d7a4164e4d9f346ec70d2bf5e6 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:35.300104 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8c512-7fe5-4a26-9147-5adf3f400425 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:35.316844 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8c512-7fe5-4a26-9147-5adf3f400425 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:35.398546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:35.409654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:35.409654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:35.409654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] PATH_INFO: `/v3/users/e0dcb6cef8774edc8bd257fbabcc8596` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:35.409654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0dcb6cef8774edc8bd257fbabcc8596)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:35.409654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:35.641969 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0dcb6cef8774edc8bd257fbabcc8596'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.645936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0dcb6cef8774edc8bd257fbabcc8596'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.667050 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0dcb6cef8774edc8bd257fbabcc8596'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.668041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0dcb6cef8774edc8bd257fbabcc8596'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.670736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:35.789456 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9b1bf62c-39e6-41c4-b596-27cc0d070466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0dcb6cef8774edc8bd257fbabcc8596'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:35.817909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1314/2639] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:35 2022] DELETE /identity/v3/users/e0dcb6cef8774edc8bd257fbabcc8596 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:36.351439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:36.353617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:36.381975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8dbee-bc86-4b94-8819-0adf48800fbc admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:36.451808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:36.452364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:36.452641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:36.452912 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] PATH_INFO: `/v3/projects/0b2e0774415b4d64a86ec47f2a7b936f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:36.454180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2e0774415b4d64a86ec47f2a7b936f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:36.456152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:36.569612 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:36.584594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2e0774415b4d64a86ec47f2a7b936f', 'request_id': 'req-84368651-dad6-441f-94ae-42bd00e8fcf7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:36.590187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2e0774415b4d64a86ec47f2a7b936f', 'request_id': 'req-84368651-dad6-441f-94ae-42bd00e8fcf7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:36.689484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2e0774415b4d64a86ec47f2a7b936f', 'request_id': 'req-84368651-dad6-441f-94ae-42bd00e8fcf7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:36.703229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] The token cache is being invalidate because project 0b2e0774415b4d64a86ec47f2a7b936f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:36.703452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:36.704188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:36.705855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1326/2640] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:36 2022] DELETE /identity/v3/projects/0b2e0774415b4d64a86ec47f2a7b936f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:36.710947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597bd6b-8c49-4da8-b570-251ea9e6180d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:36.711426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597bd6b-8c49-4da8-b570-251ea9e6180d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:36.711562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597bd6b-8c49-4da8-b570-251ea9e6180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:36.734710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-d597bd6b-8c49-4da8-b570-251ea9e6180d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:36.735944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d597bd6b-8c49-4da8-b570-251ea9e6180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:36.793143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:36.797484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:36.816497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1315/2641] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:36.825620 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f713e-ca10-4308-99ab-c2f20f798da6 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:36.826436 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f713e-ca10-4308-99ab-c2f20f798da6 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:36.826839 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f713e-ca10-4308-99ab-c2f20f798da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:36.850658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-9c5f713e-ca10-4308-99ab-c2f20f798da6 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:36.852142 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-9c5f713e-ca10-4308-99ab-c2f20f798da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:36.863107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1316/2642] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:36.868912 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f713e-ca10-4308-99ab-c2f20f798da6 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:36.880470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f713e-ca10-4308-99ab-c2f20f798da6 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:36.887206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f6df5-15a0-4a63-b37b-6a8cab7013bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:36.887958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f6df5-15a0-4a63-b37b-6a8cab7013bb admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:36.888425 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f6df5-15a0-4a63-b37b-6a8cab7013bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:36.888773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f6df5-15a0-4a63-b37b-6a8cab7013bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:36.889584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f6df5-15a0-4a63-b37b-6a8cab7013bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:36:36.889967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f6df5-15a0-4a63-b37b-6a8cab7013bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:36.891852 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f6df5-15a0-4a63-b37b-6a8cab7013bb admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:36.904400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84368651-dad6-441f-94ae-42bd00e8fcf7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:36.907749 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1317/2643] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:36:36.914060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6ea9b-be9b-456a-9311-d1f89729591f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:36.914261 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f6df5-15a0-4a63-b37b-6a8cab7013bb admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:36.915867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6ea9b-be9b-456a-9311-d1f89729591f service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:36.916353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6ea9b-be9b-456a-9311-d1f89729591f service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:36.916825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6ea9b-be9b-456a-9311-d1f89729591f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:36.919194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-53b6ea9b-be9b-456a-9311-d1f89729591f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:36.925579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f6df5-15a0-4a63-b37b-6a8cab7013bb admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:36.932773 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bda82-331c-4d24-9d14-5b839075fdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:36.933523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bda82-331c-4d24-9d14-5b839075fdc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:36.933884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bda82-331c-4d24-9d14-5b839075fdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:36.934229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bda82-331c-4d24-9d14-5b839075fdc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:36.935526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bda82-331c-4d24-9d14-5b839075fdc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:36.938345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bda82-331c-4d24-9d14-5b839075fdc1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:37.011969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6ea9b-be9b-456a-9311-d1f89729591f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:37.014689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6ea9b-be9b-456a-9311-d1f89729591f service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:37.038928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1327/2644] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:36:37.104469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1318/2645] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:36 2022] POST /identity/v3/projects => generated 389 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:37.110547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6ea9b-be9b-456a-9311-d1f89729591f service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:37.122151 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6ea9b-be9b-456a-9311-d1f89729591f service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:37.189788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543348cf-4827-4e57-aff4-011f4045f56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:37.191196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543348cf-4827-4e57-aff4-011f4045f56f admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:37.191554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543348cf-4827-4e57-aff4-011f4045f56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:37.191772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543348cf-4827-4e57-aff4-011f4045f56f admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:37.192596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543348cf-4827-4e57-aff4-011f4045f56f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:37.194443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543348cf-4827-4e57-aff4-011f4045f56f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:37.259966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bda82-331c-4d24-9d14-5b839075fdc1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:37.261919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-018bda82-331c-4d24-9d14-5b839075fdc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:37.282632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bda82-331c-4d24-9d14-5b839075fdc1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:37.288376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:37.289258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:37.289536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:37.289810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] PATH_INFO: `/v3/users/fa208dd76b074319b49f42a2a64af728` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:37.291030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa208dd76b074319b49f42a2a64af728)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:37.292952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:37.341709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1328/2646] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:37 2022] POST /identity/v3/users => generated 295 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:37.346750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543348cf-4827-4e57-aff4-011f4045f56f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:37.363651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543348cf-4827-4e57-aff4-011f4045f56f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:37.370870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cf35a-2c9b-4310-bef5-3a3423828023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:37.371644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf35a-2c9b-4310-bef5-3a3423828023 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:37.372038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf35a-2c9b-4310-bef5-3a3423828023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:37.372415 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf35a-2c9b-4310-bef5-3a3423828023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:37.373853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cf35a-2c9b-4310-bef5-3a3423828023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:37.375620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cf35a-2c9b-4310-bef5-3a3423828023 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:37.385911 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1329/2647] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:37.391090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cf35a-2c9b-4310-bef5-3a3423828023 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:37.400646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cf35a-2c9b-4310-bef5-3a3423828023 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:37.406152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa848ca-7343-4000-8e24-f5d8f8326913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:37.406821 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa848ca-7343-4000-8e24-f5d8f8326913 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:37.407138 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa848ca-7343-4000-8e24-f5d8f8326913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:37.407416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa848ca-7343-4000-8e24-f5d8f8326913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:37.408147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa848ca-7343-4000-8e24-f5d8f8326913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:37.409742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa848ca-7343-4000-8e24-f5d8f8326913 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:37.420945 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1330/2648] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:36:37.425189 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa848ca-7343-4000-8e24-f5d8f8326913 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:37.436764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa848ca-7343-4000-8e24-f5d8f8326913 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:37.441823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d7bb3-1427-48dc-b9e5-4b62db92a157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:37.442841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d7bb3-1427-48dc-b9e5-4b62db92a157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:37.443253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d7bb3-1427-48dc-b9e5-4b62db92a157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:37.443537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d7bb3-1427-48dc-b9e5-4b62db92a157 admin admin] PATH_INFO: `/v3/projects/53123eca0384467daeaeb87a770ebab8/users/4c7b32549ca74bb1b83ba787ed19253a/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:37.461051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d7bb3-1427-48dc-b9e5-4b62db92a157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53123eca0384467daeaeb87a770ebab8, user_id=4c7b32549ca74bb1b83ba787ed19253a, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:37.464120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d7bb3-1427-48dc-b9e5-4b62db92a157 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:37.534667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa208dd76b074319b49f42a2a64af728'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:37.535401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa208dd76b074319b49f42a2a64af728'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:37.554887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa208dd76b074319b49f42a2a64af728'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:37.555903 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa208dd76b074319b49f42a2a64af728'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:37.561866 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1331/2649] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:37 2022] PUT /identity/v3/projects/53123eca0384467daeaeb87a770ebab8/users/4c7b32549ca74bb1b83ba787ed19253a/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:36:37.579273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f10c99-2491-45e5-ad03-8d299ea0edc1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:37.579949 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f10c99-2491-45e5-ad03-8d299ea0edc1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:37.580235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f10c99-2491-45e5-ad03-8d299ea0edc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:37.615783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-f9f10c99-2491-45e5-ad03-8d299ea0edc1 None None] MFA Rules not processed for user `4c7b32549ca74bb1b83ba787ed19253a`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:37.617009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f9f10c99-2491-45e5-ad03-8d299ea0edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:37.626348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:37.672365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-67caf513-d721-46c0-8f81-ee7af0e9b4ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa208dd76b074319b49f42a2a64af728'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:37.690169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1319/2650] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:37 2022] DELETE /identity/v3/users/fa208dd76b074319b49f42a2a64af728 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:37.694824 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ede82-5e99-409d-9919-3462f1f02d8b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:37.695321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ede82-5e99-409d-9919-3462f1f02d8b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:37.695952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ede82-5e99-409d-9919-3462f1f02d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:37.713332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1332/2651] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:37.733966 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f10c99-2491-45e5-ad03-8d299ea0edc1 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:37.736687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f9f10c99-2491-45e5-ad03-8d299ea0edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:37.738207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e88ede82-5e99-409d-9919-3462f1f02d8b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:37.739989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e88ede82-5e99-409d-9919-3462f1f02d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:37.751675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=3,keystone:DELETE=11,keystone:SELECT=86 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:37.809341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1320/2652] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:36:37.816881 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f10c99-2491-45e5-ad03-8d299ea0edc1 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:37.822341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ede82-5e99-409d-9919-3462f1f02d8b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:37.823960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e88ede82-5e99-409d-9919-3462f1f02d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:37.876069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c4bff-31b7-4e1b-bf57-60729783a89e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:37.876879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c4bff-31b7-4e1b-bf57-60729783a89e service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:37.877242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c4bff-31b7-4e1b-bf57-60729783a89e service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:37.877589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c4bff-31b7-4e1b-bf57-60729783a89e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:37.887657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ede82-5e99-409d-9919-3462f1f02d8b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:37.892419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db7a2-31a3-4c44-b8fb-8e8d600dfd38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:37.894523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db7a2-31a3-4c44-b8fb-8e8d600dfd38 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:37.894523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db7a2-31a3-4c44-b8fb-8e8d600dfd38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:37.894523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63db7a2-31a3-4c44-b8fb-8e8d600dfd38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:37.905076 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63db7a2-31a3-4c44-b8fb-8e8d600dfd38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:37.906857 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63db7a2-31a3-4c44-b8fb-8e8d600dfd38 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:37.917709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1321/2653] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:37.946125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c4bff-31b7-4e1b-bf57-60729783a89e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:37.946125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925c4bff-31b7-4e1b-bf57-60729783a89e service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:37.957198 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1333/2654] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:36:38.284207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:38.406426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db7a2-31a3-4c44-b8fb-8e8d600dfd38 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:38.407949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d63db7a2-31a3-4c44-b8fb-8e8d600dfd38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:38.423009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63db7a2-31a3-4c44-b8fb-8e8d600dfd38 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:38.428595 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:38.429463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:38.429996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:38.430171 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] PATH_INFO: `/v3/projects/43aff8e7c37542eb9aac50045674d1bc` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:38.432059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43aff8e7c37542eb9aac50045674d1bc)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:38.434035 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:38.543357 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43aff8e7c37542eb9aac50045674d1bc', 'request_id': 'req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.552681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43aff8e7c37542eb9aac50045674d1bc', 'request_id': 'req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.616218 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43aff8e7c37542eb9aac50045674d1bc', 'request_id': 'req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.630681 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] The token cache is being invalidate because project 43aff8e7c37542eb9aac50045674d1bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:38.631043 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.631701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.633201 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1322/2655] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:38 2022] DELETE /identity/v3/projects/43aff8e7c37542eb9aac50045674d1bc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:38.637112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c4bff-31b7-4e1b-bf57-60729783a89e service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:38.649794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c4bff-31b7-4e1b-bf57-60729783a89e service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:38.716686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:38.717722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:38.718319 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:38.718870 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] PATH_INFO: `/v3/users/43d6d2e404c74f15b6ecdf61331f904d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:38.720988 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d6d2e404c74f15b6ecdf61331f904d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:38.722975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:38.933361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d6d2e404c74f15b6ecdf61331f904d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.940565 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d6d2e404c74f15b6ecdf61331f904d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.941227 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d6d2e404c74f15b6ecdf61331f904d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.941843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d6d2e404c74f15b6ecdf61331f904d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.980942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-e92fc0c1-d978-4830-a932-2bee451faf3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d6d2e404c74f15b6ecdf61331f904d'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:38.989208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:38.999596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1334/2656] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:36:38 2022] DELETE /identity/v3/users/43d6d2e404c74f15b6ecdf61331f904d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:39.441088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:39.442615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:39.456841 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69ffa3-9fcf-4134-8c19-4df96bea80c5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:39.497622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:39.516713 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:39.517645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:39.517931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:39.518210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] PATH_INFO: `/v3/projects/84953e645e1e48a0831e2244cbeb24ff` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:39.519445 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=84953e645e1e48a0831e2244cbeb24ff)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:39.521368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:39.647960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84953e645e1e48a0831e2244cbeb24ff', 'request_id': 'req-2980de11-9d96-4db0-8110-edf9d95fcd6f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:39.653855 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84953e645e1e48a0831e2244cbeb24ff', 'request_id': 'req-2980de11-9d96-4db0-8110-edf9d95fcd6f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:39.786936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84953e645e1e48a0831e2244cbeb24ff', 'request_id': 'req-2980de11-9d96-4db0-8110-edf9d95fcd6f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:39.799492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] The token cache is being invalidate because project 84953e645e1e48a0831e2244cbeb24ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:36:39.799863 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:39.800481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-2980de11-9d96-4db0-8110-edf9d95fcd6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:36:39.802019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1323/2657] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:36:39 2022] DELETE /identity/v3/projects/84953e645e1e48a0831e2244cbeb24ff => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:36:39.807355 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2f906-e309-4299-9adf-866a8dcb7508 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:39.807813 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2f906-e309-4299-9adf-866a8dcb7508 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:39.807928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2f906-e309-4299-9adf-866a8dcb7508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:39.827478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-01d2f906-e309-4299-9adf-866a8dcb7508 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:39.828982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-01d2f906-e309-4299-9adf-866a8dcb7508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:39.895459 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1335/2658] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:39.914577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe03f-9852-4f54-950b-03f30dfb0c2f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:39.914882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe03f-9852-4f54-950b-03f30dfb0c2f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:39.915224 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe03f-9852-4f54-950b-03f30dfb0c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:39.937249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-127fe03f-9852-4f54-950b-03f30dfb0c2f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:39.938635 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-127fe03f-9852-4f54-950b-03f30dfb0c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:39.947177 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1324/2659] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:39.952548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2f906-e309-4299-9adf-866a8dcb7508 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:39.962226 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2f906-e309-4299-9adf-866a8dcb7508 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:39.968348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ddfc3-ebd4-4076-98e3-2318835931d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:39.969008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ddfc3-ebd4-4076-98e3-2318835931d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:39.969292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ddfc3-ebd4-4076-98e3-2318835931d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:39.969567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ddfc3-ebd4-4076-98e3-2318835931d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:39.970287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ddfc3-ebd4-4076-98e3-2318835931d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:36:39.970622 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ddfc3-ebd4-4076-98e3-2318835931d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:39.972225 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ddfc3-ebd4-4076-98e3-2318835931d7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:39.984833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1336/2660] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:36:39.989582 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe03f-9852-4f54-950b-03f30dfb0c2f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:39.999288 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe03f-9852-4f54-950b-03f30dfb0c2f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:40.004101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836e46d-ed49-475b-9e21-b255c75ace86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:40.009282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836e46d-ed49-475b-9e21-b255c75ace86 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:40.009282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836e46d-ed49-475b-9e21-b255c75ace86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:40.009282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836e46d-ed49-475b-9e21-b255c75ace86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:40.009282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836e46d-ed49-475b-9e21-b255c75ace86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:40.009282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0836e46d-ed49-475b-9e21-b255c75ace86 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:40.098060 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1325/2661] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:36:39 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:36:40.102379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ddfc3-ebd4-4076-98e3-2318835931d7 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:40.112906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ddfc3-ebd4-4076-98e3-2318835931d7 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:40.191528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c3365-2bc9-420d-8db8-afce050515ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:40.192215 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c3365-2bc9-420d-8db8-afce050515ee admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:40.192732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c3365-2bc9-420d-8db8-afce050515ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:40.192877 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c3365-2bc9-420d-8db8-afce050515ee admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:40.193493 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c3365-2bc9-420d-8db8-afce050515ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:40.195484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c3365-2bc9-420d-8db8-afce050515ee admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:40.277471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1337/2662] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:36:40 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:36:40.280183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836e46d-ed49-475b-9e21-b255c75ace86 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:40.292111 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836e46d-ed49-475b-9e21-b255c75ace86 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:40.298611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632123-8326-4d4a-822a-8c6068ae2352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:40.299657 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632123-8326-4d4a-822a-8c6068ae2352 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:40.299948 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632123-8326-4d4a-822a-8c6068ae2352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:40.300232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632123-8326-4d4a-822a-8c6068ae2352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:40.301002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632123-8326-4d4a-822a-8c6068ae2352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:40.302672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632123-8326-4d4a-822a-8c6068ae2352 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:40.318799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1326/2663] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:40.323022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c3365-2bc9-420d-8db8-afce050515ee admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:40.335462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c3365-2bc9-420d-8db8-afce050515ee admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:40.340439 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c47139-db11-4170-8e23-8fd6d0d44010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:40.341161 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c47139-db11-4170-8e23-8fd6d0d44010 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:40.341456 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c47139-db11-4170-8e23-8fd6d0d44010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:40.341748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c47139-db11-4170-8e23-8fd6d0d44010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:40.342533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c47139-db11-4170-8e23-8fd6d0d44010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:40.344194 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c47139-db11-4170-8e23-8fd6d0d44010 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:40.355647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1338/2664] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:36:40.360054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632123-8326-4d4a-822a-8c6068ae2352 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:40.371323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632123-8326-4d4a-822a-8c6068ae2352 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:40.376964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed54fb-b164-4d27-ba00-a5c4cea0fef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:40.377832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed54fb-b164-4d27-ba00-a5c4cea0fef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:40.378118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed54fb-b164-4d27-ba00-a5c4cea0fef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:40.378399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed54fb-b164-4d27-ba00-a5c4cea0fef1 admin admin] PATH_INFO: `/v3/projects/990703a3ac71459bb47bdae1ccfbb0fe/users/141060f7d4924c82b7d1929ee0bad598/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:40.395071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed54fb-b164-4d27-ba00-a5c4cea0fef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990703a3ac71459bb47bdae1ccfbb0fe, user_id=141060f7d4924c82b7d1929ee0bad598, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:40.397766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed54fb-b164-4d27-ba00-a5c4cea0fef1 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:40.485439 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1327/2665] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:36:40 2022] PUT /identity/v3/projects/990703a3ac71459bb47bdae1ccfbb0fe/users/141060f7d4924c82b7d1929ee0bad598/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:36:40.498659 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dfc21-c5db-4afe-a88c-c45c2f1d6253 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:40.498967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dfc21-c5db-4afe-a88c-c45c2f1d6253 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:40.499253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dfc21-c5db-4afe-a88c-c45c2f1d6253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:40.526426 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-725dfc21-c5db-4afe-a88c-c45c2f1d6253 None None] MFA Rules not processed for user `141060f7d4924c82b7d1929ee0bad598`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:36:40.527653 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-725dfc21-c5db-4afe-a88c-c45c2f1d6253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:40.573594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:40.595457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1339/2666] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:36:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:36:40.608376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed54fb-b164-4d27-ba00-a5c4cea0fef1 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:36:40.610045 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d7ed54fb-b164-4d27-ba00-a5c4cea0fef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:36:40.674154 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed54fb-b164-4d27-ba00-a5c4cea0fef1 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:36:40.677945 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaed73f-1fc7-4500-bc03-4de493839a9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:36:40.678628 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaed73f-1fc7-4500-bc03-4de493839a9b service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:36:40.678915 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaed73f-1fc7-4500-bc03-4de493839a9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:36:40.679195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaed73f-1fc7-4500-bc03-4de493839a9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:36:40.688210 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaed73f-1fc7-4500-bc03-4de493839a9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:36:40.689652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaed73f-1fc7-4500-bc03-4de493839a9b service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:36:40.698769 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1328/2667] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:36:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:36:50.694649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:36:50.992126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:12.412564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dfc21-c5db-4afe-a88c-c45c2f1d6253 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:12.414042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-725dfc21-c5db-4afe-a88c-c45c2f1d6253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:12.493487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dfc21-c5db-4afe-a88c-c45c2f1d6253 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:12.498367 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:12.499296 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:12.499596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:12.499864 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] PATH_INFO: `/v3/users/4c7b32549ca74bb1b83ba787ed19253a` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:12.501745 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7b32549ca74bb1b83ba787ed19253a)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:12.504010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:12.762466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7b32549ca74bb1b83ba787ed19253a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:12.769817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7b32549ca74bb1b83ba787ed19253a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:12.770598 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7b32549ca74bb1b83ba787ed19253a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:12.771300 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7b32549ca74bb1b83ba787ed19253a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:12.872405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7b32549ca74bb1b83ba787ed19253a'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:12.897954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1340/2668] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:12 2022] DELETE /identity/v3/users/4c7b32549ca74bb1b83ba787ed19253a => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:12.902392 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87023-9a98-4fcc-8145-f75c0c4c0caa None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:12.902963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87023-9a98-4fcc-8145-f75c0c4c0caa None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:12.903072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd87023-9a98-4fcc-8145-f75c0c4c0caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:12.926755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ebd87023-9a98-4fcc-8145-f75c0c4c0caa None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:12.929214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ebd87023-9a98-4fcc-8145-f75c0c4c0caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:12.993004 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1329/2669] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:13.003174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:13.004624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:13.077513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e801a-d0fc-47df-9914-2e23bfa89372 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:13.081076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb5adf-3294-4550-863b-1edd18a01efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:13.081730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb5adf-3294-4550-863b-1edd18a01efa service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:13.082016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb5adf-3294-4550-863b-1edd18a01efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:13.082297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb5adf-3294-4550-863b-1edd18a01efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:13.092463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb5adf-3294-4550-863b-1edd18a01efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:13.093907 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb5adf-3294-4550-863b-1edd18a01efa service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:13.104431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1341/2670] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:13.531584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87023-9a98-4fcc-8145-f75c0c4c0caa None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:13.533169 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ebd87023-9a98-4fcc-8145-f75c0c4c0caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:13.546313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd87023-9a98-4fcc-8145-f75c0c4c0caa None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:13.551905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:13.552887 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:13.553238 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:13.553544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] PATH_INFO: `/v3/projects/53123eca0384467daeaeb87a770ebab8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:13.554805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53123eca0384467daeaeb87a770ebab8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:13.557013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:13.667710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53123eca0384467daeaeb87a770ebab8', 'request_id': 'req-ecab1f3b-ec66-49e1-b746-42e781e0e63f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:13.670113 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53123eca0384467daeaeb87a770ebab8', 'request_id': 'req-ecab1f3b-ec66-49e1-b746-42e781e0e63f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:13.757452 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53123eca0384467daeaeb87a770ebab8', 'request_id': 'req-ecab1f3b-ec66-49e1-b746-42e781e0e63f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:13.770458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] The token cache is being invalidate because project 53123eca0384467daeaeb87a770ebab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:13.770754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:13.772573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-ecab1f3b-ec66-49e1-b746-42e781e0e63f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:13.774175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1330/2671] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:13 2022] DELETE /identity/v3/projects/53123eca0384467daeaeb87a770ebab8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:19.070476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb5adf-3294-4550-863b-1edd18a01efa service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:19.071917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9feb5adf-3294-4550-863b-1edd18a01efa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:19.100065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:19.166027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb5adf-3294-4550-863b-1edd18a01efa service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:19.226015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:19.226689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:19.226977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:19.227257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:19.229038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:19.242033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:19.243670 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:19.253702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1342/2672] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:21.643145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99edcc-2c3a-41ca-b179-042a296c24d4 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:21.643691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99edcc-2c3a-41ca-b179-042a296c24d4 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:21.643691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99edcc-2c3a-41ca-b179-042a296c24d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:21.663868 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3a99edcc-2c3a-41ca-b179-042a296c24d4 None None] MFA Rules not processed for user `0beb4f30b5c34ab0afdbfbf46dbfede5`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:21.665463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3a99edcc-2c3a-41ca-b179-042a296c24d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:21.736358 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1331/2673] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:22.110515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:22.111928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:22.178210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39431bb-4473-42f5-b38f-e183b1545b41 service placement] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:22.182090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:22.184617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:22.184617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:22.184617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:22.185112 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:22.195377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:22.196808 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:22.204823 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1343/2674] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:23.012696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99edcc-2c3a-41ca-b179-042a296c24d4 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:23.014296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3a99edcc-2c3a-41ca-b179-042a296c24d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:23.032708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99edcc-2c3a-41ca-b179-042a296c24d4 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:23.037980 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:23.038680 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:23.042584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:23.042584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:23.042584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:23.054103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:23.056306 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:23.070268 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1332/2675] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:23.475043 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:23.476882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:23.487878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aec968-5db7-4668-8163-2188228fae31 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:23.492708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c24f89-a3b4-4d08-9cfc-b98c3bbc9f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:23.493353 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c24f89-a3b4-4d08-9cfc-b98c3bbc9f18 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:23.493633 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c24f89-a3b4-4d08-9cfc-b98c3bbc9f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:23.493908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c24f89-a3b4-4d08-9cfc-b98c3bbc9f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:23.495695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d0c24f89-a3b4-4d08-9cfc-b98c3bbc9f18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:23.530937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=100 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:23.563361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c24f89-a3b4-4d08-9cfc-b98c3bbc9f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:23.564797 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c24f89-a3b4-4d08-9cfc-b98c3bbc9f18 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:23.575535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1344/2676] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:23.953720 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:23.955253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:23.969432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1867476-b70e-47e3-a46b-6eadf4bf7453 service placement] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:23.977448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:23.978468 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:23.978814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:23.979333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] PATH_INFO: `/v3/users/141060f7d4924c82b7d1929ee0bad598` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:23.980660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=141060f7d4924c82b7d1929ee0bad598)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:23.982788 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:23.995031 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:24.166547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '141060f7d4924c82b7d1929ee0bad598'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:24.167130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '141060f7d4924c82b7d1929ee0bad598'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:24.178220 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '141060f7d4924c82b7d1929ee0bad598'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:24.179951 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '141060f7d4924c82b7d1929ee0bad598'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:24.289010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '141060f7d4924c82b7d1929ee0bad598'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:24.304698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1333/2677] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:23 2022] DELETE /identity/v3/users/141060f7d4924c82b7d1929ee0bad598 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:24.315077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fe55c-5296-4159-8da4-c3600addf180 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:24.315077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fe55c-5296-4159-8da4-c3600addf180 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:24.315077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fe55c-5296-4159-8da4-c3600addf180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:24.328588 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-5e7fe55c-5296-4159-8da4-c3600addf180 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:24.329697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5e7fe55c-5296-4159-8da4-c3600addf180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:24.402474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1345/2678] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:37:24.411992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:24.414779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:24.474217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136abff-b886-4248-b019-82b9fdef2fa3 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:24.478751 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e36a4a-c200-401c-9e58-ecda8b331f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:24.479415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e36a4a-c200-401c-9e58-ecda8b331f0f service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:24.479706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e36a4a-c200-401c-9e58-ecda8b331f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:24.479989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e36a4a-c200-401c-9e58-ecda8b331f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:24.490376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e36a4a-c200-401c-9e58-ecda8b331f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:24.492494 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e36a4a-c200-401c-9e58-ecda8b331f0f service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:24.502296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1334/2679] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:25.361219 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fe55c-5296-4159-8da4-c3600addf180 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:25.362811 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5e7fe55c-5296-4159-8da4-c3600addf180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:25.376687 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fe55c-5296-4159-8da4-c3600addf180 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:25.382143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:25.383039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:25.383470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:25.384016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] PATH_INFO: `/v3/projects/990703a3ac71459bb47bdae1ccfbb0fe` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:25.385491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=990703a3ac71459bb47bdae1ccfbb0fe)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:25.387918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:25.485715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990703a3ac71459bb47bdae1ccfbb0fe', 'request_id': 'req-2594795d-6d54-4954-977e-02cc53a275ca'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:25.493009 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990703a3ac71459bb47bdae1ccfbb0fe', 'request_id': 'req-2594795d-6d54-4954-977e-02cc53a275ca'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:25.551461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990703a3ac71459bb47bdae1ccfbb0fe', 'request_id': 'req-2594795d-6d54-4954-977e-02cc53a275ca'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:25.567880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] The token cache is being invalidate because project 990703a3ac71459bb47bdae1ccfbb0fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:25.574854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:25.574854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2594795d-6d54-4954-977e-02cc53a275ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:25.574854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1346/2680] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:25 2022] DELETE /identity/v3/projects/990703a3ac71459bb47bdae1ccfbb0fe => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:25.576725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83c8b4-d99d-45a3-83be-ab54d184c11b None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:25.577178 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83c8b4-d99d-45a3-83be-ab54d184c11b None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:25.577303 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83c8b4-d99d-45a3-83be-ab54d184c11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:25.596779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0f83c8b4-d99d-45a3-83be-ab54d184c11b None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:25.598013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0f83c8b4-d99d-45a3-83be-ab54d184c11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:25.666464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1335/2681] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:25.673914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bfe06-536c-4ca6-b615-8fdef400c3cb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:25.674201 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bfe06-536c-4ca6-b615-8fdef400c3cb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:25.674611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bfe06-536c-4ca6-b615-8fdef400c3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:25.698597 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c19bfe06-536c-4ca6-b615-8fdef400c3cb None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:25.699767 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c19bfe06-536c-4ca6-b615-8fdef400c3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:25.715587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1347/2682] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:25.719918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83c8b4-d99d-45a3-83be-ab54d184c11b None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:25.733609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83c8b4-d99d-45a3-83be-ab54d184c11b None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:25.739838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a4e28-fa66-4971-883f-72ec3b6b7f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:25.740516 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a4e28-fa66-4971-883f-72ec3b6b7f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:25.740837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a4e28-fa66-4971-883f-72ec3b6b7f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:25.741116 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a4e28-fa66-4971-883f-72ec3b6b7f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:25.742685 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a4e28-fa66-4971-883f-72ec3b6b7f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:37:25.743000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a4e28-fa66-4971-883f-72ec3b6b7f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:25.745024 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a4e28-fa66-4971-883f-72ec3b6b7f7a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:25.762481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1336/2683] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:37:25.767351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bfe06-536c-4ca6-b615-8fdef400c3cb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:25.780569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bfe06-536c-4ca6-b615-8fdef400c3cb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:25.786703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0d0b-c576-48e3-a201-c34eb9d19ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:25.789093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0d0b-c576-48e3-a201-c34eb9d19ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:25.789412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0d0b-c576-48e3-a201-c34eb9d19ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:25.789726 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e0d0b-c576-48e3-a201-c34eb9d19ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:25.790487 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e0d0b-c576-48e3-a201-c34eb9d19ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:25.792462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e0d0b-c576-48e3-a201-c34eb9d19ab8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:25.945038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1348/2684] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:25 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:37:25.977590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a4e28-fa66-4971-883f-72ec3b6b7f7a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:25.977590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a4e28-fa66-4971-883f-72ec3b6b7f7a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:25.996148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:26.103817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed38dd-2f50-4419-8473-47133a14e465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:26.104493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed38dd-2f50-4419-8473-47133a14e465 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:26.104783 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed38dd-2f50-4419-8473-47133a14e465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:26.105213 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed38dd-2f50-4419-8473-47133a14e465 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:26.105991 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed38dd-2f50-4419-8473-47133a14e465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:26.107957 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed38dd-2f50-4419-8473-47133a14e465 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:26.251712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1337/2685] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:25 2022] POST /identity/v3/users => generated 284 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:26.255719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0d0b-c576-48e3-a201-c34eb9d19ab8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:26.275964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e0d0b-c576-48e3-a201-c34eb9d19ab8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:26.278115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f4bbe-2767-4ae4-b690-ed0d806697ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:26.279105 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f4bbe-2767-4ae4-b690-ed0d806697ae admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:26.280231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f4bbe-2767-4ae4-b690-ed0d806697ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:26.280231 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f4bbe-2767-4ae4-b690-ed0d806697ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:26.281312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f4bbe-2767-4ae4-b690-ed0d806697ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:26.283923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f4bbe-2767-4ae4-b690-ed0d806697ae admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:26.302497 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed38dd-2f50-4419-8473-47133a14e465 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:26.303076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1349/2686] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:26.317037 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed38dd-2f50-4419-8473-47133a14e465 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:26.322318 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a233351-0f51-487c-b63a-1106b72b5538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:26.323051 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233351-0f51-487c-b63a-1106b72b5538 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:26.323493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233351-0f51-487c-b63a-1106b72b5538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:26.324068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233351-0f51-487c-b63a-1106b72b5538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:26.324966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a233351-0f51-487c-b63a-1106b72b5538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:26.326905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a233351-0f51-487c-b63a-1106b72b5538 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:26.340808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1338/2687] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:26.346125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f4bbe-2767-4ae4-b690-ed0d806697ae admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:26.362755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f4bbe-2767-4ae4-b690-ed0d806697ae admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:26.367975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7efcc-dbe1-45fc-93d1-2fd06fa4a77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:26.368900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7efcc-dbe1-45fc-93d1-2fd06fa4a77a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:26.369180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7efcc-dbe1-45fc-93d1-2fd06fa4a77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:26.369454 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7efcc-dbe1-45fc-93d1-2fd06fa4a77a admin admin] PATH_INFO: `/v3/projects/0280b11c02534bc2b9b81e226e0e39d6/users/6adb938a73894524ac86fa1a264ca38c/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:26.382647 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:26.386188 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7efcc-dbe1-45fc-93d1-2fd06fa4a77a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0280b11c02534bc2b9b81e226e0e39d6, user_id=6adb938a73894524ac86fa1a264ca38c, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:26.388891 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7efcc-dbe1-45fc-93d1-2fd06fa4a77a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:26.543879 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1350/2688] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:26 2022] PUT /identity/v3/projects/0280b11c02534bc2b9b81e226e0e39d6/users/6adb938a73894524ac86fa1a264ca38c/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:26.557001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd834ef-6c00-46dc-a3ab-4651689aebd3 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:26.557511 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd834ef-6c00-46dc-a3ab-4651689aebd3 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:26.558039 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd834ef-6c00-46dc-a3ab-4651689aebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:26.588931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-fbd834ef-6c00-46dc-a3ab-4651689aebd3 None None] MFA Rules not processed for user `6adb938a73894524ac86fa1a264ca38c`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:26.590423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fbd834ef-6c00-46dc-a3ab-4651689aebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:26.670542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1339/2689] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:37:26.682372 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7efcc-dbe1-45fc-93d1-2fd06fa4a77a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:26.687095 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-fad7efcc-dbe1-45fc-93d1-2fd06fa4a77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:26.783998 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7efcc-dbe1-45fc-93d1-2fd06fa4a77a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:26.788162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d2742-d53c-4464-8ee8-ffb18a169166 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9969632e796345fa9671a45cf61ede3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:26.788833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d2742-d53c-4464-8ee8-ffb18a169166 service glance] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:26.789120 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d2742-d53c-4464-8ee8-ffb18a169166 service glance] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:26.789400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d2742-d53c-4464-8ee8-ffb18a169166 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:26.799130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d2742-d53c-4464-8ee8-ffb18a169166 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:26.802134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d2742-d53c-4464-8ee8-ffb18a169166 service glance] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:26.811181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1351/2690] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:28.701651 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd834ef-6c00-46dc-a3ab-4651689aebd3 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:28.703296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fbd834ef-6c00-46dc-a3ab-4651689aebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:28.835222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd834ef-6c00-46dc-a3ab-4651689aebd3 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:28.863611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:28.865414 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:28.865911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:28.866388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] PATH_INFO: `/v3/users/f0a98f1c068843a684f6765d41cc4e49` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:28.867882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a98f1c068843a684f6765d41cc4e49)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:28.869914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:28.971737 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:29.036831 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a98f1c068843a684f6765d41cc4e49'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.038932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a98f1c068843a684f6765d41cc4e49'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.054293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a98f1c068843a684f6765d41cc4e49'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.055068 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a98f1c068843a684f6765d41cc4e49'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.122632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-98c1ee4d-f5b7-438b-89fc-4d751242b811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a98f1c068843a684f6765d41cc4e49'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.143514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1340/2691] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:28 2022] DELETE /identity/v3/users/f0a98f1c068843a684f6765d41cc4e49 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:29.387742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d2742-d53c-4464-8ee8-ffb18a169166 service glance] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:29.403533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d2742-d53c-4464-8ee8-ffb18a169166 service glance] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:29.468723 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:29.469596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:29.470081 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:29.470365 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] PATH_INFO: `/v3/projects/86c353f6ded643e6b501ec57800fafc6` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:29.471756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86c353f6ded643e6b501ec57800fafc6)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:29.477125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:29.598746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86c353f6ded643e6b501ec57800fafc6', 'request_id': 'req-4a28414b-a08f-4da9-aad6-d901fe0c3c78'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.608706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86c353f6ded643e6b501ec57800fafc6', 'request_id': 'req-4a28414b-a08f-4da9-aad6-d901fe0c3c78'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.689018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86c353f6ded643e6b501ec57800fafc6', 'request_id': 'req-4a28414b-a08f-4da9-aad6-d901fe0c3c78'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.704143 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] The token cache is being invalidate because project 86c353f6ded643e6b501ec57800fafc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:29.704505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.705145 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-4a28414b-a08f-4da9-aad6-d901fe0c3c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:29.706795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1352/2692] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:29 2022] DELETE /identity/v3/projects/86c353f6ded643e6b501ec57800fafc6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:29.712672 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67db9c-8d3f-44f3-99ee-02bcd72b7203 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:29.713180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67db9c-8d3f-44f3-99ee-02bcd72b7203 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:29.713315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67db9c-8d3f-44f3-99ee-02bcd72b7203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:29.732981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ba67db9c-8d3f-44f3-99ee-02bcd72b7203 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:29.734182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ba67db9c-8d3f-44f3-99ee-02bcd72b7203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:29.796310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1341/2693] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:29.808047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34706c4-1a27-4156-a62c-2a7a503d477f None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:29.808047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34706c4-1a27-4156-a62c-2a7a503d477f None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:29.808047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34706c4-1a27-4156-a62c-2a7a503d477f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:29.816074 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:29.828861 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e34706c4-1a27-4156-a62c-2a7a503d477f None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:29.830220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e34706c4-1a27-4156-a62c-2a7a503d477f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:29.839398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1353/2694] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:37:29.843621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67db9c-8d3f-44f3-99ee-02bcd72b7203 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:29.853371 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67db9c-8d3f-44f3-99ee-02bcd72b7203 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:29.858255 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed888b96-c83a-4d20-8ec2-75e74133ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:29.858970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed888b96-c83a-4d20-8ec2-75e74133ba30 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:29.863753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed888b96-c83a-4d20-8ec2-75e74133ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:29.863753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed888b96-c83a-4d20-8ec2-75e74133ba30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:29.863753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed888b96-c83a-4d20-8ec2-75e74133ba30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:37:29.863753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed888b96-c83a-4d20-8ec2-75e74133ba30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:29.863753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed888b96-c83a-4d20-8ec2-75e74133ba30 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:29.880165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1342/2695] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:37:29.881976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34706c4-1a27-4156-a62c-2a7a503d477f None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:29.891960 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34706c4-1a27-4156-a62c-2a7a503d477f None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:29.897819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfcdaf-c0be-4ff6-b26a-7d4084bef49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:29.898658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfcdaf-c0be-4ff6-b26a-7d4084bef49c admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:29.898954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfcdaf-c0be-4ff6-b26a-7d4084bef49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:29.899361 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfcdaf-c0be-4ff6-b26a-7d4084bef49c admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:29.900164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfcdaf-c0be-4ff6-b26a-7d4084bef49c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:29.902738 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfcdaf-c0be-4ff6-b26a-7d4084bef49c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:29.987512 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1354/2696] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:29 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:37:29.992230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed888b96-c83a-4d20-8ec2-75e74133ba30 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:30.002295 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed888b96-c83a-4d20-8ec2-75e74133ba30 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:30.060636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465724d8-7748-4b95-8989-6e14ba77d06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:30.061216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465724d8-7748-4b95-8989-6e14ba77d06b admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:30.061499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465724d8-7748-4b95-8989-6e14ba77d06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:30.061775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465724d8-7748-4b95-8989-6e14ba77d06b admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:30.062499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465724d8-7748-4b95-8989-6e14ba77d06b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:30.064411 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465724d8-7748-4b95-8989-6e14ba77d06b admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:30.140114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1343/2697] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:29 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:30.144375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfcdaf-c0be-4ff6-b26a-7d4084bef49c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:30.156364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfcdaf-c0be-4ff6-b26a-7d4084bef49c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:30.161166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46834-578b-4593-9fe6-16533dbeb507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:30.161882 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46834-578b-4593-9fe6-16533dbeb507 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:30.162162 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46834-578b-4593-9fe6-16533dbeb507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:30.162483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46834-578b-4593-9fe6-16533dbeb507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:30.163208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec46834-578b-4593-9fe6-16533dbeb507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:30.164800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec46834-578b-4593-9fe6-16533dbeb507 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:30.176425 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1355/2698] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:30.180971 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465724d8-7748-4b95-8989-6e14ba77d06b admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:30.191830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465724d8-7748-4b95-8989-6e14ba77d06b admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:30.197692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d28d42-69c6-492e-9b35-257244959b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:30.198560 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d28d42-69c6-492e-9b35-257244959b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:30.198851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d28d42-69c6-492e-9b35-257244959b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:30.199132 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d28d42-69c6-492e-9b35-257244959b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:30.199940 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d28d42-69c6-492e-9b35-257244959b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:30.201589 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d28d42-69c6-492e-9b35-257244959b12 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:30.214250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1344/2699] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:30.218535 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46834-578b-4593-9fe6-16533dbeb507 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:30.229078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46834-578b-4593-9fe6-16533dbeb507 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:30.234039 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57c7f4-3548-41f2-8e7f-ffeff90ca66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:30.234964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57c7f4-3548-41f2-8e7f-ffeff90ca66a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:30.236090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57c7f4-3548-41f2-8e7f-ffeff90ca66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:30.236090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57c7f4-3548-41f2-8e7f-ffeff90ca66a admin admin] PATH_INFO: `/v3/projects/7d70b80c295040c481bcb2f87147593f/users/146816f543e342bba35085f0fca83d8f/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:30.253495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57c7f4-3548-41f2-8e7f-ffeff90ca66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d70b80c295040c481bcb2f87147593f, user_id=146816f543e342bba35085f0fca83d8f, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:30.256860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57c7f4-3548-41f2-8e7f-ffeff90ca66a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:30.334502 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1356/2700] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:30 2022] PUT /identity/v3/projects/7d70b80c295040c481bcb2f87147593f/users/146816f543e342bba35085f0fca83d8f/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:30.345583 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4675d-d736-4d06-92a3-5fbbee5c9649 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:30.345896 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4675d-d736-4d06-92a3-5fbbee5c9649 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:30.346147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4675d-d736-4d06-92a3-5fbbee5c9649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:30.366286 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:30.374660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-66b4675d-d736-4d06-92a3-5fbbee5c9649 None None] MFA Rules not processed for user `146816f543e342bba35085f0fca83d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:30.375871 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-66b4675d-d736-4d06-92a3-5fbbee5c9649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:30.443100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1345/2701] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:30.455104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57c7f4-3548-41f2-8e7f-ffeff90ca66a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:30.456790 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cc57c7f4-3548-41f2-8e7f-ffeff90ca66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:30.531736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57c7f4-3548-41f2-8e7f-ffeff90ca66a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:30.537875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97629ea7-94ab-4497-ba47-4704d0f2c738 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:30.538569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97629ea7-94ab-4497-ba47-4704d0f2c738 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:30.538859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97629ea7-94ab-4497-ba47-4704d0f2c738 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:30.539134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97629ea7-94ab-4497-ba47-4704d0f2c738 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:30.548016 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97629ea7-94ab-4497-ba47-4704d0f2c738 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:30.549626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97629ea7-94ab-4497-ba47-4704d0f2c738 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:30.560638 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1357/2702] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:33.970566 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4675d-d736-4d06-92a3-5fbbee5c9649 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:34.034698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4675d-d736-4d06-92a3-5fbbee5c9649 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:34.039216 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:34.040194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:34.040499 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:34.040800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] PATH_INFO: `/v3/users/146816f543e342bba35085f0fca83d8f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:34.042071 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=146816f543e342bba35085f0fca83d8f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:34.044284 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:34.251690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '146816f543e342bba35085f0fca83d8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:34.252360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '146816f543e342bba35085f0fca83d8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:34.265283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '146816f543e342bba35085f0fca83d8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:34.265959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '146816f543e342bba35085f0fca83d8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:34.357135 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '146816f543e342bba35085f0fca83d8f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:34.371515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1346/2703] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:33 2022] DELETE /identity/v3/users/146816f543e342bba35085f0fca83d8f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:34.375955 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fdfbe-3c83-4cf7-b113-5269217a6e15 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:34.376452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fdfbe-3c83-4cf7-b113-5269217a6e15 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:34.376554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fdfbe-3c83-4cf7-b113-5269217a6e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:34.394503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d92fdfbe-3c83-4cf7-b113-5269217a6e15 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:34.395734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d92fdfbe-3c83-4cf7-b113-5269217a6e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:34.457329 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1358/2704] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:34.467654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:34.469056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:34.496804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:34.533966 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bf4e7-9f69-4f6e-87ae-bd0c69ef39e9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:34.537960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a4b81-2d97-47d9-902c-3e7244f2b915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:34.538692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a4b81-2d97-47d9-902c-3e7244f2b915 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:34.539960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a4b81-2d97-47d9-902c-3e7244f2b915 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:34.540289 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a4b81-2d97-47d9-902c-3e7244f2b915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:34.549742 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a4b81-2d97-47d9-902c-3e7244f2b915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:34.551150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a4b81-2d97-47d9-902c-3e7244f2b915 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:34.563322 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1347/2705] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:35.032833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fdfbe-3c83-4cf7-b113-5269217a6e15 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:35.034325 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d92fdfbe-3c83-4cf7-b113-5269217a6e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:35.046377 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fdfbe-3c83-4cf7-b113-5269217a6e15 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:35.052730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:35.052730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.052730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.052730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] PATH_INFO: `/v3/projects/7d70b80c295040c481bcb2f87147593f` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.054246 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d70b80c295040c481bcb2f87147593f)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:35.056620 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:35.084564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:35.172928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d70b80c295040c481bcb2f87147593f', 'request_id': 'req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:35.178497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d70b80c295040c481bcb2f87147593f', 'request_id': 'req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:35.254239 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d70b80c295040c481bcb2f87147593f', 'request_id': 'req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:35.266646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] The token cache is being invalidate because project 7d70b80c295040c481bcb2f87147593f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:35.266933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:35.267577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-338ccf1e-6fbb-4b43-8b0c-d7385bd0e266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:35.269381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1359/2706] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:35 2022] DELETE /identity/v3/projects/7d70b80c295040c481bcb2f87147593f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:35.274654 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289bc21-c42f-4ba1-8188-62a81eefdb96 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.275732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289bc21-c42f-4ba1-8188-62a81eefdb96 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.275846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289bc21-c42f-4ba1-8188-62a81eefdb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.296072 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-2289bc21-c42f-4ba1-8188-62a81eefdb96 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:35.297331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-2289bc21-c42f-4ba1-8188-62a81eefdb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:35.361347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1348/2707] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:35.369297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d113322-3257-4b6f-b318-7789d45de61c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.369589 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d113322-3257-4b6f-b318-7789d45de61c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.369868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d113322-3257-4b6f-b318-7789d45de61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.388290 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8d113322-3257-4b6f-b318-7789d45de61c None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:35.389506 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8d113322-3257-4b6f-b318-7789d45de61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:35.398369 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1360/2708] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:35.403441 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289bc21-c42f-4ba1-8188-62a81eefdb96 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:35.413174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289bc21-c42f-4ba1-8188-62a81eefdb96 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:35.418897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560ea20-1e0e-41b0-b5c5-0da675de47a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:35.419544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560ea20-1e0e-41b0-b5c5-0da675de47a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.419825 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560ea20-1e0e-41b0-b5c5-0da675de47a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.420096 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560ea20-1e0e-41b0-b5c5-0da675de47a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.420816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560ea20-1e0e-41b0-b5c5-0da675de47a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:37:35.421121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560ea20-1e0e-41b0-b5c5-0da675de47a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:35.422724 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560ea20-1e0e-41b0-b5c5-0da675de47a6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:35.436558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1349/2709] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:37:35.440932 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d113322-3257-4b6f-b318-7789d45de61c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:35.450914 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d113322-3257-4b6f-b318-7789d45de61c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:35.455566 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569251eb-7b28-480a-9fcb-910c3be47268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:35.456221 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569251eb-7b28-480a-9fcb-910c3be47268 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.456505 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569251eb-7b28-480a-9fcb-910c3be47268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.456779 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569251eb-7b28-480a-9fcb-910c3be47268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.457491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569251eb-7b28-480a-9fcb-910c3be47268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:35.459427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569251eb-7b28-480a-9fcb-910c3be47268 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:35.553546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1361/2710] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:35 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:37:35.558648 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560ea20-1e0e-41b0-b5c5-0da675de47a6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:35.568814 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560ea20-1e0e-41b0-b5c5-0da675de47a6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:35.628731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fb4e5-4cdd-4155-b298-d9b30d0ab113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:35.629427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fb4e5-4cdd-4155-b298-d9b30d0ab113 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.629717 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fb4e5-4cdd-4155-b298-d9b30d0ab113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.629992 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fb4e5-4cdd-4155-b298-d9b30d0ab113 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.630963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9fb4e5-4cdd-4155-b298-d9b30d0ab113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:35.633900 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9fb4e5-4cdd-4155-b298-d9b30d0ab113 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:35.757205 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1350/2711] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:35 2022] POST /identity/v3/users => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:35.762073 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569251eb-7b28-480a-9fcb-910c3be47268 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:35.772727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569251eb-7b28-480a-9fcb-910c3be47268 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:35.778077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9a8c2-6868-4099-a2e7-6d813996e409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:35.782757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9a8c2-6868-4099-a2e7-6d813996e409 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.782757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9a8c2-6868-4099-a2e7-6d813996e409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.782757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9a8c2-6868-4099-a2e7-6d813996e409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.782757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9a8c2-6868-4099-a2e7-6d813996e409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:35.782757 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9a8c2-6868-4099-a2e7-6d813996e409 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:35.792618 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1362/2712] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:35.796819 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fb4e5-4cdd-4155-b298-d9b30d0ab113 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:35.808389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fb4e5-4cdd-4155-b298-d9b30d0ab113 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:35.813228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509264f-b0b8-433a-8e24-00f20aaef962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:35.813897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509264f-b0b8-433a-8e24-00f20aaef962 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.814188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509264f-b0b8-433a-8e24-00f20aaef962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.814502 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509264f-b0b8-433a-8e24-00f20aaef962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.815234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509264f-b0b8-433a-8e24-00f20aaef962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:35.816823 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509264f-b0b8-433a-8e24-00f20aaef962 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:35.830592 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1351/2713] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:35.843360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9a8c2-6868-4099-a2e7-6d813996e409 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:35.855969 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9a8c2-6868-4099-a2e7-6d813996e409 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:35.860810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06479b16-f9ee-466b-99ab-dfcb8dc71e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:35.861699 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06479b16-f9ee-466b-99ab-dfcb8dc71e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.861985 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06479b16-f9ee-466b-99ab-dfcb8dc71e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.862262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06479b16-f9ee-466b-99ab-dfcb8dc71e40 admin admin] PATH_INFO: `/v3/projects/7ab3fd70899c40eb902cb1c73e030dd1/users/34a96ce3a2e942b4ae0b3a24819d5b1c/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.877787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06479b16-f9ee-466b-99ab-dfcb8dc71e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab3fd70899c40eb902cb1c73e030dd1, user_id=34a96ce3a2e942b4ae0b3a24819d5b1c, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:35.880520 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06479b16-f9ee-466b-99ab-dfcb8dc71e40 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:35.952961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1363/2714] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:35 2022] PUT /identity/v3/projects/7ab3fd70899c40eb902cb1c73e030dd1/users/34a96ce3a2e942b4ae0b3a24819d5b1c/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:35.958292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10012bad-d76c-4771-a7d5-04d4d109c47d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:35.958604 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10012bad-d76c-4771-a7d5-04d4d109c47d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:35.958891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10012bad-d76c-4771-a7d5-04d4d109c47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:35.977658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-10012bad-d76c-4771-a7d5-04d4d109c47d None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:35.984792 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-10012bad-d76c-4771-a7d5-04d4d109c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:36.004448 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:36.043602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1352/2715] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:36.053980 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06479b16-f9ee-466b-99ab-dfcb8dc71e40 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:36.057381 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-06479b16-f9ee-466b-99ab-dfcb8dc71e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:36.122902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06479b16-f9ee-466b-99ab-dfcb8dc71e40 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:36.126923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e206a-0603-4b04-bdba-d7f34c114e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:36.127561 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e206a-0603-4b04-bdba-d7f34c114e02 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:36.127844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e206a-0603-4b04-bdba-d7f34c114e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:36.128121 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e206a-0603-4b04-bdba-d7f34c114e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:36.139115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e206a-0603-4b04-bdba-d7f34c114e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:36.140483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e206a-0603-4b04-bdba-d7f34c114e02 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:36.151015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1364/2716] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:37.571840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d666c-3f16-4e78-b472-2446c1e20d8d None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:37.571840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d666c-3f16-4e78-b472-2446c1e20d8d None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:37.571840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d666c-3f16-4e78-b472-2446c1e20d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:37.571840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6c9d666c-3f16-4e78-b472-2446c1e20d8d None None] MFA Rules not processed for user `34a96ce3a2e942b4ae0b3a24819d5b1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:37.571840 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6c9d666c-3f16-4e78-b472-2446c1e20d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:37.638378 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1353/2717] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:37.642995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e206a-0603-4b04-bdba-d7f34c114e02 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:37.653006 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e206a-0603-4b04-bdba-d7f34c114e02 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:37.659210 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bd9fa-243a-469c-b44f-38159558c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:37.659881 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bd9fa-243a-469c-b44f-38159558c84e admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:37.660170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bd9fa-243a-469c-b44f-38159558c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:37.660461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bd9fa-243a-469c-b44f-38159558c84e admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:37.661167 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bd9fa-243a-469c-b44f-38159558c84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:37.663160 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bd9fa-243a-469c-b44f-38159558c84e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:37.895491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1365/2718] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:37 2022] POST /identity/v3/projects => generated 403 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:37.899369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d666c-3f16-4e78-b472-2446c1e20d8d None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:37.910310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d666c-3f16-4e78-b472-2446c1e20d8d None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:37.970684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ca7ed-22d5-4e94-8397-52aaa4c4d5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:37.971367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ca7ed-22d5-4e94-8397-52aaa4c4d5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:37.971650 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ca7ed-22d5-4e94-8397-52aaa4c4d5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:37.971946 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ca7ed-22d5-4e94-8397-52aaa4c4d5d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:37.972638 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ca7ed-22d5-4e94-8397-52aaa4c4d5d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:37.974587 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ca7ed-22d5-4e94-8397-52aaa4c4d5d6 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:38.077156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1354/2719] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:37 2022] POST /identity/v3/users => generated 308 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:38.081303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bd9fa-243a-469c-b44f-38159558c84e admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:38.092401 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bd9fa-243a-469c-b44f-38159558c84e admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:38.096875 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54043cd9-ae83-4ede-a040-2e2bdd139805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:38.097529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54043cd9-ae83-4ede-a040-2e2bdd139805 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:38.097812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54043cd9-ae83-4ede-a040-2e2bdd139805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:38.103242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54043cd9-ae83-4ede-a040-2e2bdd139805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:38.103242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54043cd9-ae83-4ede-a040-2e2bdd139805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:38.103242 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54043cd9-ae83-4ede-a040-2e2bdd139805 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:38.111859 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1366/2720] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:38.116388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ca7ed-22d5-4e94-8397-52aaa4c4d5d6 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:38.126794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ca7ed-22d5-4e94-8397-52aaa4c4d5d6 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:38.131460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607be92-544d-4e25-9b9c-e7b3adbeb0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:38.132402 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607be92-544d-4e25-9b9c-e7b3adbeb0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:38.132686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607be92-544d-4e25-9b9c-e7b3adbeb0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:38.132964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607be92-544d-4e25-9b9c-e7b3adbeb0f9 admin admin] PATH_INFO: `/v3/projects/3dd747356f08439ebe6abdb1178853a4/users/b3db5807b7164bba89856e79bddb3998/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:38.148117 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607be92-544d-4e25-9b9c-e7b3adbeb0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd747356f08439ebe6abdb1178853a4, user_id=b3db5807b7164bba89856e79bddb3998, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:38.151461 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607be92-544d-4e25-9b9c-e7b3adbeb0f9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:38.306903 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1355/2721] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:38 2022] PUT /identity/v3/projects/3dd747356f08439ebe6abdb1178853a4/users/b3db5807b7164bba89856e79bddb3998/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:38.311289 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54043cd9-ae83-4ede-a040-2e2bdd139805 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:38.321442 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54043cd9-ae83-4ede-a040-2e2bdd139805 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:38.333621 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:38.384852 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d3ff0-1b8b-45bb-924a-e7fd1a9b3a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:38.385819 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d3ff0-1b8b-45bb-924a-e7fd1a9b3a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:38.386104 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d3ff0-1b8b-45bb-924a-e7fd1a9b3a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:38.386390 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d3ff0-1b8b-45bb-924a-e7fd1a9b3a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:38.387150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d3ff0-1b8b-45bb-924a-e7fd1a9b3a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:38.388800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d3ff0-1b8b-45bb-924a-e7fd1a9b3a60 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:38.398608 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1367/2722] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:38.403601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607be92-544d-4e25-9b9c-e7b3adbeb0f9 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:38.413908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607be92-544d-4e25-9b9c-e7b3adbeb0f9 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:38.419693 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd6553-5890-4e03-a80d-a82d1647f1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:38.420581 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6553-5890-4e03-a80d-a82d1647f1ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:38.420864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6553-5890-4e03-a80d-a82d1647f1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:38.421165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6553-5890-4e03-a80d-a82d1647f1ce admin admin] PATH_INFO: `/v3/projects/3dd747356f08439ebe6abdb1178853a4/users/b3db5807b7164bba89856e79bddb3998/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:38.423222 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd6553-5890-4e03-a80d-a82d1647f1ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd747356f08439ebe6abdb1178853a4, user_id=b3db5807b7164bba89856e79bddb3998, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:38.425865 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd6553-5890-4e03-a80d-a82d1647f1ce admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:38.438341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1356/2723] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:38 2022] PUT /identity/v3/projects/3dd747356f08439ebe6abdb1178853a4/users/b3db5807b7164bba89856e79bddb3998/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:39.750217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a62542-fed0-4b06-8d42-09592fa5a133 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:39.750851 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a62542-fed0-4b06-8d42-09592fa5a133 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:39.750953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a62542-fed0-4b06-8d42-09592fa5a133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:39.792141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-26a62542-fed0-4b06-8d42-09592fa5a133 None None] MFA Rules not processed for user `b3db5807b7164bba89856e79bddb3998`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:39.792141 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-26a62542-fed0-4b06-8d42-09592fa5a133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:39.866878 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1368/2724] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:39 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:37:41.262939 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd6553-5890-4e03-a80d-a82d1647f1ce admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:41.265202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-a6fd6553-5890-4e03-a80d-a82d1647f1ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:41.281187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:41.358481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd6553-5890-4e03-a80d-a82d1647f1ce admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:41.364670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51775d5b-50f7-4146-9be0-5d19dfb7133e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:41.365339 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51775d5b-50f7-4146-9be0-5d19dfb7133e service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:41.365631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51775d5b-50f7-4146-9be0-5d19dfb7133e service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:41.365914 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51775d5b-50f7-4146-9be0-5d19dfb7133e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:41.367922 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-51775d5b-50f7-4146-9be0-5d19dfb7133e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:41.385562 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51775d5b-50f7-4146-9be0-5d19dfb7133e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:41.387692 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51775d5b-50f7-4146-9be0-5d19dfb7133e service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:41.404180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1357/2725] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:41.563462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a62542-fed0-4b06-8d42-09592fa5a133 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:41.628403 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a62542-fed0-4b06-8d42-09592fa5a133 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:41.632965 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:41.633838 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:41.634117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:41.634963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] PATH_INFO: `/v3/users/34a96ce3a2e942b4ae0b3a24819d5b1c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:41.636532 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a96ce3a2e942b4ae0b3a24819d5b1c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:41.638471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:41.833574 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:41.840260 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a96ce3a2e942b4ae0b3a24819d5b1c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:41.847497 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a96ce3a2e942b4ae0b3a24819d5b1c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:41.848200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a96ce3a2e942b4ae0b3a24819d5b1c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:41.848812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a96ce3a2e942b4ae0b3a24819d5b1c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:41.916995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a96ce3a2e942b4ae0b3a24819d5b1c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:41.938477 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1369/2726] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:41 2022] DELETE /identity/v3/users/34a96ce3a2e942b4ae0b3a24819d5b1c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:37:42.175373 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51775d5b-50f7-4146-9be0-5d19dfb7133e service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:42.187249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51775d5b-50f7-4146-9be0-5d19dfb7133e service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:42.246305 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:42.247770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:42.248055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:42.248406 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] PATH_INFO: `/v3/projects/7ab3fd70899c40eb902cb1c73e030dd1` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:42.249524 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab3fd70899c40eb902cb1c73e030dd1)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:42.251580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:42.352067 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab3fd70899c40eb902cb1c73e030dd1', 'request_id': 'req-5559d623-b8df-446c-9611-c57b90658900'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.358226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab3fd70899c40eb902cb1c73e030dd1', 'request_id': 'req-5559d623-b8df-446c-9611-c57b90658900'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.459429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab3fd70899c40eb902cb1c73e030dd1', 'request_id': 'req-5559d623-b8df-446c-9611-c57b90658900'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.471700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] The token cache is being invalidate because project 7ab3fd70899c40eb902cb1c73e030dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:42.471988 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.472649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.475636 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1358/2727] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:42 2022] DELETE /identity/v3/projects/7ab3fd70899c40eb902cb1c73e030dd1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:37:42.478886 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:42.480402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:42.494795 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35445f-15ed-4ec7-8c31-3daf52d64811 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:42.568274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:42.571902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:42.571902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:42.571902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] PATH_INFO: `/v3/users/b3db5807b7164bba89856e79bddb3998` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:42.571902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3db5807b7164bba89856e79bddb3998)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:42.572781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:42.765148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3db5807b7164bba89856e79bddb3998'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.771922 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3db5807b7164bba89856e79bddb3998'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.772640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3db5807b7164bba89856e79bddb3998'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.773256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3db5807b7164bba89856e79bddb3998'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.819021 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3102bc1e-2553-4fe6-98f8-46e2b32d1eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3db5807b7164bba89856e79bddb3998'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:42.837336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1370/2728] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:42 2022] DELETE /identity/v3/users/b3db5807b7164bba89856e79bddb3998 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:43.119955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:43.120102 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:43.133660 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559d623-b8df-446c-9611-c57b90658900 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:43.160950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:43.194580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:43.195444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:43.195728 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:43.196007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] PATH_INFO: `/v3/projects/3dd747356f08439ebe6abdb1178853a4` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:43.197193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dd747356f08439ebe6abdb1178853a4)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:43.199137 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:43.311584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dd747356f08439ebe6abdb1178853a4', 'request_id': 'req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:43.317931 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dd747356f08439ebe6abdb1178853a4', 'request_id': 'req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:43.397174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dd747356f08439ebe6abdb1178853a4', 'request_id': 'req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:43.410185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] The token cache is being invalidate because project 3dd747356f08439ebe6abdb1178853a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:43.410498 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:43.411138 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5b7fd6f7-ec3d-4002-b6ea-5c5b9b1e1f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:43.412834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1359/2729] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:43 2022] DELETE /identity/v3/projects/3dd747356f08439ebe6abdb1178853a4 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:37:43.418671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99498ea2-41ae-42f1-87ed-13310b2a1e9a None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:43.418967 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99498ea2-41ae-42f1-87ed-13310b2a1e9a None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:43.419249 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99498ea2-41ae-42f1-87ed-13310b2a1e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:43.437810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-99498ea2-41ae-42f1-87ed-13310b2a1e9a None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:43.439055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-99498ea2-41ae-42f1-87ed-13310b2a1e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:43.471266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:43.500001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1371/2730] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:37:43.507665 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc0a44-38ba-475f-8e8f-7e57ec4d9def None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:43.507955 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc0a44-38ba-475f-8e8f-7e57ec4d9def None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:43.508230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc0a44-38ba-475f-8e8f-7e57ec4d9def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:43.527879 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-3abc0a44-38ba-475f-8e8f-7e57ec4d9def None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:43.529073 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3abc0a44-38ba-475f-8e8f-7e57ec4d9def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:43.544550 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1360/2731] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:43.549679 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99498ea2-41ae-42f1-87ed-13310b2a1e9a None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:43.563256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99498ea2-41ae-42f1-87ed-13310b2a1e9a None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:43.568557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9215eaa-dce3-4246-bd86-e4e518f2873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:43.575832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9215eaa-dce3-4246-bd86-e4e518f2873b admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:43.575832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9215eaa-dce3-4246-bd86-e4e518f2873b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:43.575832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9215eaa-dce3-4246-bd86-e4e518f2873b admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:43.575832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9215eaa-dce3-4246-bd86-e4e518f2873b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:37:43.575832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9215eaa-dce3-4246-bd86-e4e518f2873b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:43.575832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9215eaa-dce3-4246-bd86-e4e518f2873b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:43.592871 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1372/2732] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:37:43.597352 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc0a44-38ba-475f-8e8f-7e57ec4d9def None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:43.611158 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc0a44-38ba-475f-8e8f-7e57ec4d9def None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:43.617182 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efffa-fd49-4906-90e1-658d668d3af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:43.617856 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efffa-fd49-4906-90e1-658d668d3af3 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:43.618146 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efffa-fd49-4906-90e1-658d668d3af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:43.618447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efffa-fd49-4906-90e1-658d668d3af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:43.627617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09efffa-fd49-4906-90e1-658d668d3af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:43.627617 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09efffa-fd49-4906-90e1-658d668d3af3 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:43.757430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1361/2733] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:43 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:43.758753 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9215eaa-dce3-4246-bd86-e4e518f2873b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:43.769400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9215eaa-dce3-4246-bd86-e4e518f2873b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:43.832077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a011fb-e190-4663-9aa2-352df0fb8b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:43.832794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a011fb-e190-4663-9aa2-352df0fb8b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:43.833080 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a011fb-e190-4663-9aa2-352df0fb8b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:43.833364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a011fb-e190-4663-9aa2-352df0fb8b55 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:43.834055 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a011fb-e190-4663-9aa2-352df0fb8b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:43.836107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a011fb-e190-4663-9aa2-352df0fb8b55 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:43.904882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efffa-fd49-4906-90e1-658d668d3af3 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:43.906642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e09efffa-fd49-4906-90e1-658d668d3af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:43.918567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1373/2734] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:43 2022] POST /identity/v3/users => generated 282 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:43.922579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a011fb-e190-4663-9aa2-352df0fb8b55 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:43.932836 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a011fb-e190-4663-9aa2-352df0fb8b55 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:43.938444 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4710b4-dc0e-402f-8444-5ecd0f93c11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:43.945035 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4710b4-dc0e-402f-8444-5ecd0f93c11f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:43.945331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4710b4-dc0e-402f-8444-5ecd0f93c11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:43.945616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4710b4-dc0e-402f-8444-5ecd0f93c11f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:43.946396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4710b4-dc0e-402f-8444-5ecd0f93c11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:43.948624 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4710b4-dc0e-402f-8444-5ecd0f93c11f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:43.964772 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1374/2735] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:37:43.966601 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efffa-fd49-4906-90e1-658d668d3af3 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:43.968756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4710b4-dc0e-402f-8444-5ecd0f93c11f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:43.970528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681219db-bdc3-4d29-bd1e-ea944dda7e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:43.971206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681219db-bdc3-4d29-bd1e-ea944dda7e57 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:43.971496 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681219db-bdc3-4d29-bd1e-ea944dda7e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:43.971775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681219db-bdc3-4d29-bd1e-ea944dda7e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:43.973577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-681219db-bdc3-4d29-bd1e-ea944dda7e57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:43.979197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4710b4-dc0e-402f-8444-5ecd0f93c11f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:43.984155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e4aa4-547a-4b29-ac95-8b2c0488e1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:43.984815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e4aa4-547a-4b29-ac95-8b2c0488e1de admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:43.985107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e4aa4-547a-4b29-ac95-8b2c0488e1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:43.985389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e4aa4-547a-4b29-ac95-8b2c0488e1de admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:43.986117 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e4aa4-547a-4b29-ac95-8b2c0488e1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:43.987803 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e4aa4-547a-4b29-ac95-8b2c0488e1de admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:43.998812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1375/2736] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:37:44.002973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e4aa4-547a-4b29-ac95-8b2c0488e1de admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:44.013567 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e4aa4-547a-4b29-ac95-8b2c0488e1de admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:44.018377 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:44.018810 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba669c1-c45f-492d-9310-ad5c69013dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:44.019694 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba669c1-c45f-492d-9310-ad5c69013dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:44.019983 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba669c1-c45f-492d-9310-ad5c69013dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:44.020263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba669c1-c45f-492d-9310-ad5c69013dcb admin admin] PATH_INFO: `/v3/projects/2a52ed555a1a4242ad91fe75f88042a8/users/510e801c53b24ac0a5602646ed259aa0/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:44.036500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba669c1-c45f-492d-9310-ad5c69013dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a52ed555a1a4242ad91fe75f88042a8, user_id=510e801c53b24ac0a5602646ed259aa0, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:44.039263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba669c1-c45f-492d-9310-ad5c69013dcb admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:44.047580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681219db-bdc3-4d29-bd1e-ea944dda7e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:44.049027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681219db-bdc3-4d29-bd1e-ea944dda7e57 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:44.059929 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1362/2737] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:37:44.094842 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1376/2738] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:44 2022] PUT /identity/v3/projects/2a52ed555a1a4242ad91fe75f88042a8/users/510e801c53b24ac0a5602646ed259aa0/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:37:44.108140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed2ef9-c94b-43cc-8047-37643057d2e7 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:44.108140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed2ef9-c94b-43cc-8047-37643057d2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:44.108140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed2ef9-c94b-43cc-8047-37643057d2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:44.134488 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-15ed2ef9-c94b-43cc-8047-37643057d2e7 None None] MFA Rules not processed for user `510e801c53b24ac0a5602646ed259aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:44.136278 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-15ed2ef9-c94b-43cc-8047-37643057d2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:44.202008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1363/2739] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:37:44.206709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba669c1-c45f-492d-9310-ad5c69013dcb admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:44.271058 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba669c1-c45f-492d-9310-ad5c69013dcb admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:44.276982 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6c25f-d5ce-487b-b176-01d827765c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:44.277654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6c25f-d5ce-487b-b176-01d827765c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:44.277936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6c25f-d5ce-487b-b176-01d827765c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:44.278216 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6c25f-d5ce-487b-b176-01d827765c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:44.278947 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6c25f-d5ce-487b-b176-01d827765c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:44.280950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6c25f-d5ce-487b-b176-01d827765c3f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:44.290860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:44.362038 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1377/2740] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:44 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:44.363520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed2ef9-c94b-43cc-8047-37643057d2e7 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:44.374805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed2ef9-c94b-43cc-8047-37643057d2e7 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:44.434214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99985a-c47a-46cf-9637-80d58c2d7aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:44.434950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99985a-c47a-46cf-9637-80d58c2d7aea admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:44.435271 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99985a-c47a-46cf-9637-80d58c2d7aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:44.435462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99985a-c47a-46cf-9637-80d58c2d7aea admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:44.436168 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99985a-c47a-46cf-9637-80d58c2d7aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:44.438081 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99985a-c47a-46cf-9637-80d58c2d7aea admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:44.529105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1364/2741] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:44 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:37:44.533175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6c25f-d5ce-487b-b176-01d827765c3f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:44.544154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6c25f-d5ce-487b-b176-01d827765c3f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:44.550092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e994b4-1a98-45ca-899d-832cb520c510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:44.550805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e994b4-1a98-45ca-899d-832cb520c510 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:44.551093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e994b4-1a98-45ca-899d-832cb520c510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:44.551371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e994b4-1a98-45ca-899d-832cb520c510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:44.552107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e994b4-1a98-45ca-899d-832cb520c510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:44.553739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e994b4-1a98-45ca-899d-832cb520c510 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:44.566564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1378/2742] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:44.576918 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99985a-c47a-46cf-9637-80d58c2d7aea admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:44.587103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99985a-c47a-46cf-9637-80d58c2d7aea admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:44.592920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156f579-df97-4b33-997e-dacb301f3bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:44.593804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156f579-df97-4b33-997e-dacb301f3bae admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:44.594115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156f579-df97-4b33-997e-dacb301f3bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:44.594391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156f579-df97-4b33-997e-dacb301f3bae admin admin] PATH_INFO: `/v3/projects/10e7b24980924f84a7428f16826fd2df/users/e5db70079b0f4162ab0616b8b50cc764/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:44.609473 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156f579-df97-4b33-997e-dacb301f3bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e7b24980924f84a7428f16826fd2df, user_id=e5db70079b0f4162ab0616b8b50cc764, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:44.612287 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156f579-df97-4b33-997e-dacb301f3bae admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:44.701869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1365/2743] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:44 2022] PUT /identity/v3/projects/10e7b24980924f84a7428f16826fd2df/users/e5db70079b0f4162ab0616b8b50cc764/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:44.706037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e994b4-1a98-45ca-899d-832cb520c510 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:44.716950 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e994b4-1a98-45ca-899d-832cb520c510 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:44.783534 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03955c02-f9a2-4fb0-a9c5-c800d4a921dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:44.784220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03955c02-f9a2-4fb0-a9c5-c800d4a921dd admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:44.784605 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03955c02-f9a2-4fb0-a9c5-c800d4a921dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:44.784923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03955c02-f9a2-4fb0-a9c5-c800d4a921dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:44.785684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03955c02-f9a2-4fb0-a9c5-c800d4a921dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:44.787312 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03955c02-f9a2-4fb0-a9c5-c800d4a921dd admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:44.798008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1379/2744] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:44.802181 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156f579-df97-4b33-997e-dacb301f3bae admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:44.814888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156f579-df97-4b33-997e-dacb301f3bae admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:44.819788 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880d09d-c338-41ae-8b08-a69ff161ab0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:44.820687 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880d09d-c338-41ae-8b08-a69ff161ab0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:44.820974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880d09d-c338-41ae-8b08-a69ff161ab0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:44.821254 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880d09d-c338-41ae-8b08-a69ff161ab0c admin admin] PATH_INFO: `/v3/projects/10e7b24980924f84a7428f16826fd2df/users/e5db70079b0f4162ab0616b8b50cc764/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:44.824124 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880d09d-c338-41ae-8b08-a69ff161ab0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e7b24980924f84a7428f16826fd2df, user_id=e5db70079b0f4162ab0616b8b50cc764, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:44.826858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880d09d-c338-41ae-8b08-a69ff161ab0c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:44.837823 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1366/2745] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:44 2022] PUT /identity/v3/projects/10e7b24980924f84a7428f16826fd2df/users/e5db70079b0f4162ab0616b8b50cc764/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:44.849926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860d1b2-b0b5-4077-9d3d-3a38d12dbf1b None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:44.850273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860d1b2-b0b5-4077-9d3d-3a38d12dbf1b None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:44.850533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860d1b2-b0b5-4077-9d3d-3a38d12dbf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:44.879375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d860d1b2-b0b5-4077-9d3d-3a38d12dbf1b None None] MFA Rules not processed for user `e5db70079b0f4162ab0616b8b50cc764`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:44.880604 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d860d1b2-b0b5-4077-9d3d-3a38d12dbf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:44.953369 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1380/2746] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:44.965188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880d09d-c338-41ae-8b08-a69ff161ab0c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:44.967658 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7880d09d-c338-41ae-8b08-a69ff161ab0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:44.991898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:45.032727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880d09d-c338-41ae-8b08-a69ff161ab0c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:45.036553 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad0728-64ee-4672-bc2f-d0f6c8f2de69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:45.040331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad0728-64ee-4672-bc2f-d0f6c8f2de69 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:45.040331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad0728-64ee-4672-bc2f-d0f6c8f2de69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:45.040331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad0728-64ee-4672-bc2f-d0f6c8f2de69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:45.048365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad0728-64ee-4672-bc2f-d0f6c8f2de69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:45.049816 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad0728-64ee-4672-bc2f-d0f6c8f2de69 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:45.060483 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1367/2747] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:45.923245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860d1b2-b0b5-4077-9d3d-3a38d12dbf1b None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:45.992159 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860d1b2-b0b5-4077-9d3d-3a38d12dbf1b None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:45.997660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:45.998582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:45.998873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:45.999151 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] PATH_INFO: `/v3/users/510e801c53b24ac0a5602646ed259aa0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:46.000368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] RBAC: Authorizing `identity:delete_user(user_id=510e801c53b24ac0a5602646ed259aa0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:46.002398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:46.007508 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:46.283364 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '510e801c53b24ac0a5602646ed259aa0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:46.293748 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '510e801c53b24ac0a5602646ed259aa0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:46.295826 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '510e801c53b24ac0a5602646ed259aa0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:46.296732 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '510e801c53b24ac0a5602646ed259aa0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:46.363942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '510e801c53b24ac0a5602646ed259aa0'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:46.393480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1381/2748] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:45 2022] DELETE /identity/v3/users/510e801c53b24ac0a5602646ed259aa0 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:46.395230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdb3b8-7527-4494-b203-dcadf5453bba None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:46.401125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdb3b8-7527-4494-b203-dcadf5453bba None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:46.401125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccdb3b8-7527-4494-b203-dcadf5453bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:46.415774 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-1ccdb3b8-7527-4494-b203-dcadf5453bba None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:46.417805 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1ccdb3b8-7527-4494-b203-dcadf5453bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:46.505970 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1368/2749] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:46.512824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:46.514303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:46.586676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc651d0-5b84-4692-a034-f5e2065760db admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:46.591273 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c12eb-e2f4-4290-ad26-f926a0ad36ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:46.591939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c12eb-e2f4-4290-ad26-f926a0ad36ef service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:46.592228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c12eb-e2f4-4290-ad26-f926a0ad36ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:46.592511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c12eb-e2f4-4290-ad26-f926a0ad36ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:46.604163 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c12eb-e2f4-4290-ad26-f926a0ad36ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:46.605301 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c12eb-e2f4-4290-ad26-f926a0ad36ef service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:46.616044 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1382/2750] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:47.090916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccdb3b8-7527-4494-b203-dcadf5453bba None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:47.092554 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1ccdb3b8-7527-4494-b203-dcadf5453bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:47.107221 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccdb3b8-7527-4494-b203-dcadf5453bba None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:47.112932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:47.114434 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:47.114691 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:47.115032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] PATH_INFO: `/v3/projects/2a52ed555a1a4242ad91fe75f88042a8` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:47.116461 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a52ed555a1a4242ad91fe75f88042a8)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:47.118239 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:47.221246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a52ed555a1a4242ad91fe75f88042a8', 'request_id': 'req-b211379f-9ff6-45ce-808e-0f542f16cc03'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.227016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a52ed555a1a4242ad91fe75f88042a8', 'request_id': 'req-b211379f-9ff6-45ce-808e-0f542f16cc03'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.294296 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a52ed555a1a4242ad91fe75f88042a8', 'request_id': 'req-b211379f-9ff6-45ce-808e-0f542f16cc03'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.312321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] The token cache is being invalidate because project 2a52ed555a1a4242ad91fe75f88042a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:47.312613 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.313280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.314936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1369/2751] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:47 2022] DELETE /identity/v3/projects/2a52ed555a1a4242ad91fe75f88042a8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:47.319010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c12eb-e2f4-4290-ad26-f926a0ad36ef service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:47.329197 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c12eb-e2f4-4290-ad26-f926a0ad36ef service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:47.388746 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:47.389702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:47.390036 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:47.390348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] PATH_INFO: `/v3/users/e5db70079b0f4162ab0616b8b50cc764` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:47.392053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5db70079b0f4162ab0616b8b50cc764)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:47.394069 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:47.529491 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:47.622880 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5db70079b0f4162ab0616b8b50cc764'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.633573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5db70079b0f4162ab0616b8b50cc764'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.635412 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5db70079b0f4162ab0616b8b50cc764'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.636405 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5db70079b0f4162ab0616b8b50cc764'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.716747 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-788d17d4-8952-418a-9b17-1560b69f8c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5db70079b0f4162ab0616b8b50cc764'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:47.735135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1383/2752] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:47 2022] DELETE /identity/v3/users/e5db70079b0f4162ab0616b8b50cc764 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:48.289545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:48.291810 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:48.311446 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211379f-9ff6-45ce-808e-0f542f16cc03 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:48.357330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:48.370642 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:48.378227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:48.378227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:48.378227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] PATH_INFO: `/v3/projects/10e7b24980924f84a7428f16826fd2df` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:48.378227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e7b24980924f84a7428f16826fd2df)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:48.378227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:48.529362 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e7b24980924f84a7428f16826fd2df', 'request_id': 'req-af08aee5-67f2-402c-a8b8-28880fa606d9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:48.534858 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e7b24980924f84a7428f16826fd2df', 'request_id': 'req-af08aee5-67f2-402c-a8b8-28880fa606d9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:48.602048 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e7b24980924f84a7428f16826fd2df', 'request_id': 'req-af08aee5-67f2-402c-a8b8-28880fa606d9'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:48.614775 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] The token cache is being invalidate because project 10e7b24980924f84a7428f16826fd2df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:48.615059 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:48.615712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-af08aee5-67f2-402c-a8b8-28880fa606d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:48.617277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1370/2753] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:48 2022] DELETE /identity/v3/projects/10e7b24980924f84a7428f16826fd2df => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:37:48.622711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f0a96-c6ca-40d6-87f0-3f9338b9f491 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:48.623246 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f0a96-c6ca-40d6-87f0-3f9338b9f491 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:48.623402 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f0a96-c6ca-40d6-87f0-3f9338b9f491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:48.642867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-d38f0a96-c6ca-40d6-87f0-3f9338b9f491 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:48.644071 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-d38f0a96-c6ca-40d6-87f0-3f9338b9f491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:48.703714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1384/2754] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:48.711596 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccaf20-399b-4ac7-8a3a-f79dec26cd37 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:48.711882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccaf20-399b-4ac7-8a3a-f79dec26cd37 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:48.712163 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccaf20-399b-4ac7-8a3a-f79dec26cd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:48.731057 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ddccaf20-399b-4ac7-8a3a-f79dec26cd37 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:48.732242 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ddccaf20-399b-4ac7-8a3a-f79dec26cd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:48.742503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1371/2755] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:48.752585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f0a96-c6ca-40d6-87f0-3f9338b9f491 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:48.762954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f0a96-c6ca-40d6-87f0-3f9338b9f491 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:48.767564 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e65cef-df24-4fd6-ac2b-ff6f297768bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:48.768220 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e65cef-df24-4fd6-ac2b-ff6f297768bd admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:48.768503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e65cef-df24-4fd6-ac2b-ff6f297768bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:48.768781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e65cef-df24-4fd6-ac2b-ff6f297768bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:48.769533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e65cef-df24-4fd6-ac2b-ff6f297768bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80603) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:37:48.769841 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e65cef-df24-4fd6-ac2b-ff6f297768bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:48.771502 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e65cef-df24-4fd6-ac2b-ff6f297768bd admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:48.783172 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1385/2756] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:37:48.787389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccaf20-399b-4ac7-8a3a-f79dec26cd37 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:48.797769 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccaf20-399b-4ac7-8a3a-f79dec26cd37 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:48.802803 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcc440-67ed-4d24-8a3a-05f3905ba5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:48.803462 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcc440-67ed-4d24-8a3a-05f3905ba5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:48.803748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcc440-67ed-4d24-8a3a-05f3905ba5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:48.804024 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fcc440-67ed-4d24-8a3a-05f3905ba5c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:48.804716 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcc440-67ed-4d24-8a3a-05f3905ba5c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:48.807727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fcc440-67ed-4d24-8a3a-05f3905ba5c8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:48.920864 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1372/2757] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:48 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:48.925185 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e65cef-df24-4fd6-ac2b-ff6f297768bd admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:48.937268 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e65cef-df24-4fd6-ac2b-ff6f297768bd admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:48.999855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5ab8-04b5-426a-b76d-899dcd6a367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:49.000560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5ab8-04b5-426a-b76d-899dcd6a367a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:49.000855 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5ab8-04b5-426a-b76d-899dcd6a367a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:49.001134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5ab8-04b5-426a-b76d-899dcd6a367a admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:49.001854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e5ab8-04b5-426a-b76d-899dcd6a367a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:49.003828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e5ab8-04b5-426a-b76d-899dcd6a367a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:49.112438 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1386/2758] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:48 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:49.116703 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcc440-67ed-4d24-8a3a-05f3905ba5c8 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:49.127710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fcc440-67ed-4d24-8a3a-05f3905ba5c8 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:49.132308 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c5922-e5bd-4f9a-9023-fceff993ee73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:49.132984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c5922-e5bd-4f9a-9023-fceff993ee73 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:49.133274 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c5922-e5bd-4f9a-9023-fceff993ee73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:49.133545 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c5922-e5bd-4f9a-9023-fceff993ee73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:49.134280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c5922-e5bd-4f9a-9023-fceff993ee73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:49.135938 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c5922-e5bd-4f9a-9023-fceff993ee73 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:49.147103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1373/2759] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:49.152395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5ab8-04b5-426a-b76d-899dcd6a367a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:49.162903 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5ab8-04b5-426a-b76d-899dcd6a367a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:49.168297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6f966-18a2-4b79-a971-2ac4ec01610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:49.168971 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f966-18a2-4b79-a971-2ac4ec01610f admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:49.169253 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f966-18a2-4b79-a971-2ac4ec01610f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:49.169527 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6f966-18a2-4b79-a971-2ac4ec01610f admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:49.170248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6f966-18a2-4b79-a971-2ac4ec01610f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:49.171925 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6f966-18a2-4b79-a971-2ac4ec01610f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:49.186431 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1387/2760] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:49.192034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c5922-e5bd-4f9a-9023-fceff993ee73 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:49.205002 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c5922-e5bd-4f9a-9023-fceff993ee73 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:49.209967 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb314a74-4de6-4ac9-88d6-c16943717b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:49.210937 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb314a74-4de6-4ac9-88d6-c16943717b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:49.211179 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb314a74-4de6-4ac9-88d6-c16943717b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:49.211454 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb314a74-4de6-4ac9-88d6-c16943717b1f admin admin] PATH_INFO: `/v3/projects/b37a6146b9664ade8041365865695a0d/users/c99ee4d1cd834842927886cf4702fb60/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:49.226950 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb314a74-4de6-4ac9-88d6-c16943717b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37a6146b9664ade8041365865695a0d, user_id=c99ee4d1cd834842927886cf4702fb60, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:49.230519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb314a74-4de6-4ac9-88d6-c16943717b1f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:49.317388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1374/2761] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:49 2022] PUT /identity/v3/projects/b37a6146b9664ade8041365865695a0d/users/c99ee4d1cd834842927886cf4702fb60/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:49.329368 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582f710-1482-4c1b-b073-363b2a95f7d9 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:49.329666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582f710-1482-4c1b-b073-363b2a95f7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:49.329946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582f710-1482-4c1b-b073-363b2a95f7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:49.349486 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:49.359918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-e582f710-1482-4c1b-b073-363b2a95f7d9 None None] MFA Rules not processed for user `c99ee4d1cd834842927886cf4702fb60`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:49.361147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-e582f710-1482-4c1b-b073-363b2a95f7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:49.427973 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1388/2762] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:49.432877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb314a74-4de6-4ac9-88d6-c16943717b1f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:49.497644 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb314a74-4de6-4ac9-88d6-c16943717b1f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:49.502790 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dcfd9-7609-4bee-9810-2728e870985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:49.503465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dcfd9-7609-4bee-9810-2728e870985c admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:49.503753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dcfd9-7609-4bee-9810-2728e870985c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:49.504034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dcfd9-7609-4bee-9810-2728e870985c admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:49.504731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dcfd9-7609-4bee-9810-2728e870985c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:49.506671 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dcfd9-7609-4bee-9810-2728e870985c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:49.580725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:49.588899 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1375/2763] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:49 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:37:49.592939 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582f710-1482-4c1b-b073-363b2a95f7d9 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:49.604684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582f710-1482-4c1b-b073-363b2a95f7d9 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:49.675082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41949bfd-dfe8-4709-90ad-eb5d6498e478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:49.675768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41949bfd-dfe8-4709-90ad-eb5d6498e478 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:49.676053 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41949bfd-dfe8-4709-90ad-eb5d6498e478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:49.676332 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41949bfd-dfe8-4709-90ad-eb5d6498e478 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:49.677037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41949bfd-dfe8-4709-90ad-eb5d6498e478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:49.679028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41949bfd-dfe8-4709-90ad-eb5d6498e478 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:49.780440 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1389/2764] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:49 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:37:49.784951 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dcfd9-7609-4bee-9810-2728e870985c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:49.796255 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dcfd9-7609-4bee-9810-2728e870985c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:49.802143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348514a-68df-4c0c-bede-432dff6c52e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:49.802143 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348514a-68df-4c0c-bede-432dff6c52e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:49.802579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348514a-68df-4c0c-bede-432dff6c52e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:49.802684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348514a-68df-4c0c-bede-432dff6c52e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:49.803417 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348514a-68df-4c0c-bede-432dff6c52e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:49.805055 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348514a-68df-4c0c-bede-432dff6c52e7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:49.816786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1376/2765] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:49.820928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41949bfd-dfe8-4709-90ad-eb5d6498e478 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:49.831652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41949bfd-dfe8-4709-90ad-eb5d6498e478 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:49.838537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24670464-affe-406e-a539-bf12d33a9bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:49.839430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24670464-affe-406e-a539-bf12d33a9bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:49.839716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24670464-affe-406e-a539-bf12d33a9bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:49.839996 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24670464-affe-406e-a539-bf12d33a9bb5 admin admin] PATH_INFO: `/v3/projects/1b214ec296bb4eaf9c1e7b2a3453f7da/users/8a606df2964240f1af27ffbaa46c15eb/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:49.855398 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24670464-affe-406e-a539-bf12d33a9bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b214ec296bb4eaf9c1e7b2a3453f7da, user_id=8a606df2964240f1af27ffbaa46c15eb, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:49.857990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24670464-affe-406e-a539-bf12d33a9bb5 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:49.928674 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1390/2766] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:49 2022] PUT /identity/v3/projects/1b214ec296bb4eaf9c1e7b2a3453f7da/users/8a606df2964240f1af27ffbaa46c15eb/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:49.933152 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348514a-68df-4c0c-bede-432dff6c52e7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:49.943702 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348514a-68df-4c0c-bede-432dff6c52e7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:50.007014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d56d95-e297-4f5f-a1c5-25f7fb8f2c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:50.007690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d56d95-e297-4f5f-a1c5-25f7fb8f2c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:50.007974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d56d95-e297-4f5f-a1c5-25f7fb8f2c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:50.008250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d56d95-e297-4f5f-a1c5-25f7fb8f2c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:50.008981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d56d95-e297-4f5f-a1c5-25f7fb8f2c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:50.011321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d56d95-e297-4f5f-a1c5-25f7fb8f2c70 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:50.028666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1377/2767] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:37:50.033499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24670464-affe-406e-a539-bf12d33a9bb5 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:50.045688 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24670464-affe-406e-a539-bf12d33a9bb5 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:50.050525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3fe71-a9d3-44ef-a243-c2ab2e8b955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:50.051411 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3fe71-a9d3-44ef-a243-c2ab2e8b955d admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:50.051695 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3fe71-a9d3-44ef-a243-c2ab2e8b955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:50.052115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3fe71-a9d3-44ef-a243-c2ab2e8b955d admin admin] PATH_INFO: `/v3/projects/1b214ec296bb4eaf9c1e7b2a3453f7da/users/8a606df2964240f1af27ffbaa46c15eb/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:50.054101 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3fe71-a9d3-44ef-a243-c2ab2e8b955d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b214ec296bb4eaf9c1e7b2a3453f7da, user_id=8a606df2964240f1af27ffbaa46c15eb, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:50.057630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3fe71-a9d3-44ef-a243-c2ab2e8b955d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:50.068631 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1391/2768] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:50 2022] PUT /identity/v3/projects/1b214ec296bb4eaf9c1e7b2a3453f7da/users/8a606df2964240f1af27ffbaa46c15eb/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:50.079807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b8f6-92ec-41b2-bba6-f4713c63f87f None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:50.080108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b8f6-92ec-41b2-bba6-f4713c63f87f None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:50.080409 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b8f6-92ec-41b2-bba6-f4713c63f87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:50.108014 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-8054b8f6-92ec-41b2-bba6-f4713c63f87f None None] MFA Rules not processed for user `8a606df2964240f1af27ffbaa46c15eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:50.109249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8054b8f6-92ec-41b2-bba6-f4713c63f87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:50.178753 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1378/2769] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:50.192664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3fe71-a9d3-44ef-a243-c2ab2e8b955d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:50.194530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-60d3fe71-a9d3-44ef-a243-c2ab2e8b955d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:50.258063 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3fe71-a9d3-44ef-a243-c2ab2e8b955d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:50.262524 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067814c1-92b9-4a96-97b9-2cd8439d06cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:50.263193 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067814c1-92b9-4a96-97b9-2cd8439d06cc service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:50.263479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067814c1-92b9-4a96-97b9-2cd8439d06cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:50.263752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067814c1-92b9-4a96-97b9-2cd8439d06cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:50.270200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:50.274132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067814c1-92b9-4a96-97b9-2cd8439d06cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:50.276894 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067814c1-92b9-4a96-97b9-2cd8439d06cc service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:50.287660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1392/2770] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:51.373926 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054b8f6-92ec-41b2-bba6-f4713c63f87f None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:51.383453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054b8f6-92ec-41b2-bba6-f4713c63f87f None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:51.388776 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e71a23-db98-4594-9a1b-81bb6a01b5b1 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a606df2964240f1af27ffbaa46c15eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b214ec296bb4eaf9c1e7b2a3453f7da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:51.389437 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e71a23-db98-4594-9a1b-81bb6a01b5b1 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:51.389731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e71a23-db98-4594-9a1b-81bb6a01b5b1 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:51.390009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e71a23-db98-4594-9a1b-81bb6a01b5b1 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:51.390781 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e71a23-db98-4594-9a1b-81bb6a01b5b1 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:37:51.391098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e71a23-db98-4594-9a1b-81bb6a01b5b1 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:51.392704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e71a23-db98-4594-9a1b-81bb6a01b5b1 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:51.405341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1379/2771] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:37:51.409463 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067814c1-92b9-4a96-97b9-2cd8439d06cc service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:51.419065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067814c1-92b9-4a96-97b9-2cd8439d06cc service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:51.423665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938faf91-4c80-4ab8-b7f4-ca010cbb0421 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a606df2964240f1af27ffbaa46c15eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b214ec296bb4eaf9c1e7b2a3453f7da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:51.424351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938faf91-4c80-4ab8-b7f4-ca010cbb0421 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:51.424642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938faf91-4c80-4ab8-b7f4-ca010cbb0421 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:51.424924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938faf91-4c80-4ab8-b7f4-ca010cbb0421 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:51.425637 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938faf91-4c80-4ab8-b7f4-ca010cbb0421 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:51.427560 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938faf91-4c80-4ab8-b7f4-ca010cbb0421 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:51.523780 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1393/2772] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:51 2022] POST /identity/v3/projects => generated 355 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:51.528522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e71a23-db98-4594-9a1b-81bb6a01b5b1 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:51.538507 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e71a23-db98-4594-9a1b-81bb6a01b5b1 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:51.590491 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:51.597122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a606df2964240f1af27ffbaa46c15eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b214ec296bb4eaf9c1e7b2a3453f7da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:51.597787 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:51.598080 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:51.598370 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:51.599235 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:37:51.599577 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:51.601226 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:51.613408 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1380/2773] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:37:51.656909 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938faf91-4c80-4ab8-b7f4-ca010cbb0421 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:51.674681 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938faf91-4c80-4ab8-b7f4-ca010cbb0421 tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:51.679483 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a606df2964240f1af27ffbaa46c15eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b214ec296bb4eaf9c1e7b2a3453f7da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:51.680297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:51.680809 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:51.681090 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] PATH_INFO: `/v3/projects/aa3e3ada69ef4e058419243e49d5768b` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:51.682897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aa3e3ada69ef4e058419243e49d5768b)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:51.684917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:51.839416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3e3ada69ef4e058419243e49d5768b', 'request_id': 'req-f349ed8e-71c4-47ad-acab-bab744eb24fc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:51.845173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3e3ada69ef4e058419243e49d5768b', 'request_id': 'req-f349ed8e-71c4-47ad-acab-bab744eb24fc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:51.910805 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3e3ada69ef4e058419243e49d5768b', 'request_id': 'req-f349ed8e-71c4-47ad-acab-bab744eb24fc'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:51.923410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] The token cache is being invalidate because project aa3e3ada69ef4e058419243e49d5768b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:51.923697 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:51.924352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f349ed8e-71c4-47ad-acab-bab744eb24fc tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:51.925900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1394/2774] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:51 2022] DELETE /identity/v3/projects/aa3e3ada69ef4e058419243e49d5768b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:52.491609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:52.493512 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:52.575008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b380795-73a0-4ebe-865a-f3e80c2a8aac tempest-VolumeQuotaClassesTest-1988939197 tempest-VolumeQuotaClassesTest-1988939197-project-admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:52.637668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:52.638591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:52.638892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:52.639157 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] PATH_INFO: `/v3/users/c99ee4d1cd834842927886cf4702fb60` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:52.640379 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c99ee4d1cd834842927886cf4702fb60)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:52.642423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:52.827727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c99ee4d1cd834842927886cf4702fb60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:52.828388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c99ee4d1cd834842927886cf4702fb60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:52.842893 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c99ee4d1cd834842927886cf4702fb60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:52.843621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c99ee4d1cd834842927886cf4702fb60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:52.927936 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c99ee4d1cd834842927886cf4702fb60'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:52.942697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1381/2775] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:52 2022] DELETE /identity/v3/users/c99ee4d1cd834842927886cf4702fb60 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:37:52.946744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d8cf5-c8ea-4223-9b89-0816cd4988fb None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:52.947248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d8cf5-c8ea-4223-9b89-0816cd4988fb None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:52.947344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d8cf5-c8ea-4223-9b89-0816cd4988fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:52.965323 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-499d8cf5-c8ea-4223-9b89-0816cd4988fb None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:52.966579 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-499d8cf5-c8ea-4223-9b89-0816cd4988fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:53.025833 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1395/2776] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:53.035934 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:53.037333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:53.056886 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:53.096156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d92c1-4e63-40f6-9a70-22efcc2fb190 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:53.099771 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8e160-d439-4448-a396-872708012f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:53.100413 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8e160-d439-4448-a396-872708012f2d service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:53.100698 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8e160-d439-4448-a396-872708012f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:53.100975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8e160-d439-4448-a396-872708012f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:53.110758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8e160-d439-4448-a396-872708012f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:53.112202 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8e160-d439-4448-a396-872708012f2d service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:53.122573 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1382/2777] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:53.541051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d8cf5-c8ea-4223-9b89-0816cd4988fb None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:53.543107 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-499d8cf5-c8ea-4223-9b89-0816cd4988fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:53.559513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d8cf5-c8ea-4223-9b89-0816cd4988fb None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:53.564272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:53.565262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:53.565555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:53.565800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] PATH_INFO: `/v3/projects/b37a6146b9664ade8041365865695a0d` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:53.567089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37a6146b9664ade8041365865695a0d)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:53.568946 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:53.696391 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37a6146b9664ade8041365865695a0d', 'request_id': 'req-321ba9ba-57ff-494c-b0db-d69b9e080fd3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:53.697572 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:53.703461 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37a6146b9664ade8041365865695a0d', 'request_id': 'req-321ba9ba-57ff-494c-b0db-d69b9e080fd3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:53.842363 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37a6146b9664ade8041365865695a0d', 'request_id': 'req-321ba9ba-57ff-494c-b0db-d69b9e080fd3'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:53.854106 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] The token cache is being invalidate because project b37a6146b9664ade8041365865695a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:53.854395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:53.855087 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:53.857286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1396/2778] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:53 2022] DELETE /identity/v3/projects/b37a6146b9664ade8041365865695a0d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:53.861279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8e160-d439-4448-a396-872708012f2d service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:53.871718 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8e160-d439-4448-a396-872708012f2d service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:53.933192 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:53.934119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:53.934399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:53.934709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] PATH_INFO: `/v3/users/8a606df2964240f1af27ffbaa46c15eb` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:53.935892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a606df2964240f1af27ffbaa46c15eb)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:53.937818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:54.187986 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a606df2964240f1af27ffbaa46c15eb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:54.188639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a606df2964240f1af27ffbaa46c15eb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:54.201404 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a606df2964240f1af27ffbaa46c15eb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:54.202097 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a606df2964240f1af27ffbaa46c15eb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:54.293722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-3f240b15-c9b7-48a7-b880-03ddbd3b3707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a606df2964240f1af27ffbaa46c15eb'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:54.306838 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1383/2779] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:37:53 2022] DELETE /identity/v3/users/8a606df2964240f1af27ffbaa46c15eb => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:54.816195 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:54.817840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:54.832703 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ba9ba-57ff-494c-b0db-d69b9e080fd3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:54.896890 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:54.897800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:54.900127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:54.900127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] PATH_INFO: `/v3/projects/1b214ec296bb4eaf9c1e7b2a3453f7da` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:54.900127 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b214ec296bb4eaf9c1e7b2a3453f7da)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:54.901526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:55.020678 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b214ec296bb4eaf9c1e7b2a3453f7da', 'request_id': 'req-217a1250-0861-46cd-add5-6a66eaa089b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:55.026378 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b214ec296bb4eaf9c1e7b2a3453f7da', 'request_id': 'req-217a1250-0861-46cd-add5-6a66eaa089b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:55.111061 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b214ec296bb4eaf9c1e7b2a3453f7da', 'request_id': 'req-217a1250-0861-46cd-add5-6a66eaa089b8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:55.124782 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] The token cache is being invalidate because project 1b214ec296bb4eaf9c1e7b2a3453f7da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:37:55.125091 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:55.125744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-217a1250-0861-46cd-add5-6a66eaa089b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:37:55.127379 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1397/2780] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:37:54 2022] DELETE /identity/v3/projects/1b214ec296bb4eaf9c1e7b2a3453f7da => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:55.132376 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca689e-93fd-4fde-bbc1-118c1ac29384 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.139217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca689e-93fd-4fde-bbc1-118c1ac29384 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.139217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca689e-93fd-4fde-bbc1-118c1ac29384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.151003 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-0aca689e-93fd-4fde-bbc1-118c1ac29384 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:55.152292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0aca689e-93fd-4fde-bbc1-118c1ac29384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:55.188211 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:55.216020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1384/2781] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:55.224136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78ff44-a3e7-4a6a-88a7-74aaa00c7223 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.224432 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78ff44-a3e7-4a6a-88a7-74aaa00c7223 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.224714 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78ff44-a3e7-4a6a-88a7-74aaa00c7223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.247959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-cb78ff44-a3e7-4a6a-88a7-74aaa00c7223 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:55.249180 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-cb78ff44-a3e7-4a6a-88a7-74aaa00c7223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:55.257173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1398/2782] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:55.261727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca689e-93fd-4fde-bbc1-118c1ac29384 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:55.271748 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca689e-93fd-4fde-bbc1-118c1ac29384 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:55.276400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24abd3b-9c11-46e6-8f8e-3f2f4de5fcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:55.277664 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24abd3b-9c11-46e6-8f8e-3f2f4de5fcea admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.277949 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24abd3b-9c11-46e6-8f8e-3f2f4de5fcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.278334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24abd3b-9c11-46e6-8f8e-3f2f4de5fcea admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.279230 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24abd3b-9c11-46e6-8f8e-3f2f4de5fcea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:37:55.279495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24abd3b-9c11-46e6-8f8e-3f2f4de5fcea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:55.281150 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24abd3b-9c11-46e6-8f8e-3f2f4de5fcea admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:55.294005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1385/2783] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 12:37:55.298347 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78ff44-a3e7-4a6a-88a7-74aaa00c7223 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:55.307902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78ff44-a3e7-4a6a-88a7-74aaa00c7223 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:55.312794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe38dda-33c9-4f20-a3fb-b20c8bc0ff43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:55.313467 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe38dda-33c9-4f20-a3fb-b20c8bc0ff43 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.313750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe38dda-33c9-4f20-a3fb-b20c8bc0ff43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.314024 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe38dda-33c9-4f20-a3fb-b20c8bc0ff43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.314754 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe38dda-33c9-4f20-a3fb-b20c8bc0ff43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:55.316670 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe38dda-33c9-4f20-a3fb-b20c8bc0ff43 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:55.421794 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1399/2784] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:55 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:55.426088 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24abd3b-9c11-46e6-8f8e-3f2f4de5fcea admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:55.436518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24abd3b-9c11-46e6-8f8e-3f2f4de5fcea admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:55.494029 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b34313-a097-4eec-bd9c-b20ffe57652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:55.494727 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b34313-a097-4eec-bd9c-b20ffe57652f admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.495012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b34313-a097-4eec-bd9c-b20ffe57652f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.495290 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b34313-a097-4eec-bd9c-b20ffe57652f admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.496207 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b34313-a097-4eec-bd9c-b20ffe57652f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:55.498121 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b34313-a097-4eec-bd9c-b20ffe57652f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:55.580008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1386/2785] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:55 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:55.584387 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe38dda-33c9-4f20-a3fb-b20c8bc0ff43 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:55.595684 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe38dda-33c9-4f20-a3fb-b20c8bc0ff43 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:55.600306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0bc11-0b7e-46d7-833b-b1e54f4bdd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:55.600990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0bc11-0b7e-46d7-833b-b1e54f4bdd80 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.601277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0bc11-0b7e-46d7-833b-b1e54f4bdd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.601556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0bc11-0b7e-46d7-833b-b1e54f4bdd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.602287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0bc11-0b7e-46d7-833b-b1e54f4bdd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:55.603918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0bc11-0b7e-46d7-833b-b1e54f4bdd80 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:55.615241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1400/2786] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:55.619257 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b34313-a097-4eec-bd9c-b20ffe57652f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:55.629837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b34313-a097-4eec-bd9c-b20ffe57652f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:55.634557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901c907-8a1d-4cb0-b822-b963717d3bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:55.635213 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c907-8a1d-4cb0-b822-b963717d3bed admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.635501 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c907-8a1d-4cb0-b822-b963717d3bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.635781 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901c907-8a1d-4cb0-b822-b963717d3bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.637144 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901c907-8a1d-4cb0-b822-b963717d3bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:55.638767 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901c907-8a1d-4cb0-b822-b963717d3bed admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:55.648878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1387/2787] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:55.653176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0bc11-0b7e-46d7-833b-b1e54f4bdd80 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:55.663288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0bc11-0b7e-46d7-833b-b1e54f4bdd80 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:55.668642 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510443b5-a3a8-486f-a061-3e2e9c08c35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:55.669667 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510443b5-a3a8-486f-a061-3e2e9c08c35d admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.669953 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510443b5-a3a8-486f-a061-3e2e9c08c35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.670230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510443b5-a3a8-486f-a061-3e2e9c08c35d admin admin] PATH_INFO: `/v3/projects/d59b1bab37224711b576880eca2297be/users/3f1e565309324da58709c22047a641f0/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.686758 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510443b5-a3a8-486f-a061-3e2e9c08c35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d59b1bab37224711b576880eca2297be, user_id=3f1e565309324da58709c22047a641f0, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:55.690277 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510443b5-a3a8-486f-a061-3e2e9c08c35d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:55.780640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1401/2788] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:55 2022] PUT /identity/v3/projects/d59b1bab37224711b576880eca2297be/users/3f1e565309324da58709c22047a641f0/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:37:55.791833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4c9b5-930f-45b9-80e8-5f70ac7c97bc None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.792128 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4c9b5-930f-45b9-80e8-5f70ac7c97bc None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.792412 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4c9b5-930f-45b9-80e8-5f70ac7c97bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.820312 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-eaa4c9b5-930f-45b9-80e8-5f70ac7c97bc None None] MFA Rules not processed for user `3f1e565309324da58709c22047a641f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:55.821537 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-eaa4c9b5-930f-45b9-80e8-5f70ac7c97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:55.886922 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1388/2789] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:55.891725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510443b5-a3a8-486f-a061-3e2e9c08c35d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:55.898187 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:55.958957 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510443b5-a3a8-486f-a061-3e2e9c08c35d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:55.964175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda18f35-ac93-4802-b756-7b6c23e3bb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:55.964954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda18f35-ac93-4802-b756-7b6c23e3bb9a admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:55.965245 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda18f35-ac93-4802-b756-7b6c23e3bb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:55.965517 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda18f35-ac93-4802-b756-7b6c23e3bb9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:55.966206 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda18f35-ac93-4802-b756-7b6c23e3bb9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:55.968147 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda18f35-ac93-4802-b756-7b6c23e3bb9a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:56.077898 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1402/2790] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:55 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:37:56.082252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4c9b5-930f-45b9-80e8-5f70ac7c97bc None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:56.093830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4c9b5-930f-45b9-80e8-5f70ac7c97bc None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:56.124022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:56.151204 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac79e96-82d2-4004-af18-39c99da0747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:56.152492 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac79e96-82d2-4004-af18-39c99da0747e admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:56.152861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac79e96-82d2-4004-af18-39c99da0747e admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:56.153292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac79e96-82d2-4004-af18-39c99da0747e admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:56.154006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac79e96-82d2-4004-af18-39c99da0747e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:56.156109 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac79e96-82d2-4004-af18-39c99da0747e admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:56.311916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1389/2791] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:56 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:56.315977 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda18f35-ac93-4802-b756-7b6c23e3bb9a admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:56.327389 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda18f35-ac93-4802-b756-7b6c23e3bb9a admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:56.332059 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da72a28-8e78-4d43-9aec-9604c626b8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:56.332725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da72a28-8e78-4d43-9aec-9604c626b8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:56.340015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da72a28-8e78-4d43-9aec-9604c626b8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:56.340015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da72a28-8e78-4d43-9aec-9604c626b8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:56.340015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da72a28-8e78-4d43-9aec-9604c626b8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:56.340015 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da72a28-8e78-4d43-9aec-9604c626b8b3 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:56.347665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1403/2792] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:56.351782 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac79e96-82d2-4004-af18-39c99da0747e admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:56.362964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac79e96-82d2-4004-af18-39c99da0747e admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:56.367557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c934c83-0dcf-4275-a44c-474c2f774778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:56.368208 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c934c83-0dcf-4275-a44c-474c2f774778 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:56.368490 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c934c83-0dcf-4275-a44c-474c2f774778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:56.368766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c934c83-0dcf-4275-a44c-474c2f774778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:56.369484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c934c83-0dcf-4275-a44c-474c2f774778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:56.371834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c934c83-0dcf-4275-a44c-474c2f774778 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:56.381732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1390/2793] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:56.386937 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da72a28-8e78-4d43-9aec-9604c626b8b3 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:56.397239 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da72a28-8e78-4d43-9aec-9604c626b8b3 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:56.404362 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdf771-1e07-43c2-964d-befed4aa6ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:56.405258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdf771-1e07-43c2-964d-befed4aa6ed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:56.405539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdf771-1e07-43c2-964d-befed4aa6ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:56.405812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdf771-1e07-43c2-964d-befed4aa6ed6 admin admin] PATH_INFO: `/v3/projects/e5021de967244c8f9baf3155cc93f1e3/users/66c247a1a01f4a1eb5910ab627ff07e2/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:56.421441 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafdf771-1e07-43c2-964d-befed4aa6ed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5021de967244c8f9baf3155cc93f1e3, user_id=66c247a1a01f4a1eb5910ab627ff07e2, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:56.424176 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafdf771-1e07-43c2-964d-befed4aa6ed6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:56.482520 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1404/2794] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:56 2022] PUT /identity/v3/projects/e5021de967244c8f9baf3155cc93f1e3/users/66c247a1a01f4a1eb5910ab627ff07e2/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:56.492974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0737b-5049-4563-816e-0700dfb992fe None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:56.499469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0737b-5049-4563-816e-0700dfb992fe None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:56.499469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0737b-5049-4563-816e-0700dfb992fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:56.521523 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-34c0737b-5049-4563-816e-0700dfb992fe None None] MFA Rules not processed for user `66c247a1a01f4a1eb5910ab627ff07e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:56.522769 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-34c0737b-5049-4563-816e-0700dfb992fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:56.588531 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1391/2795] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:56.593770 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdf771-1e07-43c2-964d-befed4aa6ed6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:56.657048 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdf771-1e07-43c2-964d-befed4aa6ed6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:56.661817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0ee21-98dd-41ac-9954-b07f9da393f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:56.662500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0ee21-98dd-41ac-9954-b07f9da393f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:56.662787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0ee21-98dd-41ac-9954-b07f9da393f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:56.663067 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0ee21-98dd-41ac-9954-b07f9da393f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:56.663764 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0ee21-98dd-41ac-9954-b07f9da393f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:56.665654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0ee21-98dd-41ac-9954-b07f9da393f2 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:56.804042 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1405/2796] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:37:56 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:56.809291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0737b-5049-4563-816e-0700dfb992fe None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:56.819621 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0737b-5049-4563-816e-0700dfb992fe None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:56.878423 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27d59a-e72e-4edd-9652-4e831c612698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:56.879193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d59a-e72e-4edd-9652-4e831c612698 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:56.879475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d59a-e72e-4edd-9652-4e831c612698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:56.879746 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d59a-e72e-4edd-9652-4e831c612698 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:56.880433 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27d59a-e72e-4edd-9652-4e831c612698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:56.882374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27d59a-e72e-4edd-9652-4e831c612698 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:56.965273 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1392/2797] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:37:56 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:37:56.966509 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0ee21-98dd-41ac-9954-b07f9da393f2 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:56.977761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0ee21-98dd-41ac-9954-b07f9da393f2 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:56.984139 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370fe18-b01a-4988-a5ca-6c364aae4371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:56.985034 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370fe18-b01a-4988-a5ca-6c364aae4371 admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:56.985430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370fe18-b01a-4988-a5ca-6c364aae4371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:56.985812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370fe18-b01a-4988-a5ca-6c364aae4371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:56.986781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370fe18-b01a-4988-a5ca-6c364aae4371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:56.988593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370fe18-b01a-4988-a5ca-6c364aae4371 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:57.001354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1406/2798] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:57.005618 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27d59a-e72e-4edd-9652-4e831c612698 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:57.016993 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27d59a-e72e-4edd-9652-4e831c612698 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:57.030019 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdf052-bdae-456d-9e58-4afda8c656c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:57.030975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdf052-bdae-456d-9e58-4afda8c656c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:57.031256 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdf052-bdae-456d-9e58-4afda8c656c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:57.031529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdf052-bdae-456d-9e58-4afda8c656c8 admin admin] PATH_INFO: `/v3/projects/4e65b9036a8f4296b688c119e37c4d8e/users/547b9a48a7c241e8b9b9e755211aef90/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:57.048010 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdf052-bdae-456d-9e58-4afda8c656c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e65b9036a8f4296b688c119e37c4d8e, user_id=547b9a48a7c241e8b9b9e755211aef90, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:57.050808 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdf052-bdae-456d-9e58-4afda8c656c8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:57.129008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1393/2799] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:57 2022] PUT /identity/v3/projects/4e65b9036a8f4296b688c119e37c4d8e/users/547b9a48a7c241e8b9b9e755211aef90/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:37:57.133012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370fe18-b01a-4988-a5ca-6c364aae4371 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:57.144116 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370fe18-b01a-4988-a5ca-6c364aae4371 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:57.171001 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:57.201656 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d0eb8-e9c3-4281-a67f-6a81a16bf69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:57.202333 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d0eb8-e9c3-4281-a67f-6a81a16bf69c admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:57.202651 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d0eb8-e9c3-4281-a67f-6a81a16bf69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:57.202934 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d0eb8-e9c3-4281-a67f-6a81a16bf69c admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:57.203666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d0eb8-e9c3-4281-a67f-6a81a16bf69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:57.205270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d0eb8-e9c3-4281-a67f-6a81a16bf69c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:57.215705 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1407/2800] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:37:57.220189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdf052-bdae-456d-9e58-4afda8c656c8 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:57.231413 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdf052-bdae-456d-9e58-4afda8c656c8 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:57.236147 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e81ef-ba2d-4aa5-9aa5-e61d0b4c39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:57.237033 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e81ef-ba2d-4aa5-9aa5-e61d0b4c39f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:57.237316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e81ef-ba2d-4aa5-9aa5-e61d0b4c39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:57.237599 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e81ef-ba2d-4aa5-9aa5-e61d0b4c39f7 admin admin] PATH_INFO: `/v3/projects/4e65b9036a8f4296b688c119e37c4d8e/users/547b9a48a7c241e8b9b9e755211aef90/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:57.239588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e81ef-ba2d-4aa5-9aa5-e61d0b4c39f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e65b9036a8f4296b688c119e37c4d8e, user_id=547b9a48a7c241e8b9b9e755211aef90, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:57.242271 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e81ef-ba2d-4aa5-9aa5-e61d0b4c39f7 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:57.244645 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:37:57.253919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1394/2801] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:37:57 2022] PUT /identity/v3/projects/4e65b9036a8f4296b688c119e37c4d8e/users/547b9a48a7c241e8b9b9e755211aef90/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:37:57.264492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d169-8137-4af7-a4db-f55ed4c37ed5 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:57.264784 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d169-8137-4af7-a4db-f55ed4c37ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:57.265452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d169-8137-4af7-a4db-f55ed4c37ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:57.291751 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-c004d169-8137-4af7-a4db-f55ed4c37ed5 None None] MFA Rules not processed for user `547b9a48a7c241e8b9b9e755211aef90`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:37:57.293592 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c004d169-8137-4af7-a4db-f55ed4c37ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:57.363883 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1408/2802] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:37:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:37:57.374313 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e81ef-ba2d-4aa5-9aa5-e61d0b4c39f7 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:57.376090 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-896e81ef-ba2d-4aa5-9aa5-e61d0b4c39f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:57.434709 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e81ef-ba2d-4aa5-9aa5-e61d0b4c39f7 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:57.438335 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8ea87-ed86-4719-bb66-0d8e15d17f57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:57.439012 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ea87-ed86-4719-bb66-0d8e15d17f57 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:57.439298 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ea87-ed86-4719-bb66-0d8e15d17f57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:57.439575 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ea87-ed86-4719-bb66-0d8e15d17f57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:57.449438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8ea87-ed86-4719-bb66-0d8e15d17f57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:57.450916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8ea87-ed86-4719-bb66-0d8e15d17f57 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:57.460640 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1395/2803] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:37:58.412416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d169-8137-4af7-a4db-f55ed4c37ed5 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:37:58.413860 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-c004d169-8137-4af7-a4db-f55ed4c37ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:37:58.426763 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d169-8137-4af7-a4db-f55ed4c37ed5 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:37:58.430297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99e84c-1e5a-412b-af85-9e3740991d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:37:58.431217 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99e84c-1e5a-412b-af85-9e3740991d9b service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:37:58.431499 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99e84c-1e5a-412b-af85-9e3740991d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:37:58.431778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99e84c-1e5a-412b-af85-9e3740991d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:37:58.490135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99e84c-1e5a-412b-af85-9e3740991d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:37:58.491641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99e84c-1e5a-412b-af85-9e3740991d9b service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:37:58.500490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1409/2804] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:37:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:38:07.456199 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:07.805228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8ea87-ed86-4719-bb66-0d8e15d17f57 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:07.814661 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8ea87-ed86-4719-bb66-0d8e15d17f57 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:07.818505 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8efe1-51b5-4290-8588-83de5617be7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:07.819162 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8efe1-51b5-4290-8588-83de5617be7b service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:07.819447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8efe1-51b5-4290-8588-83de5617be7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:07.819734 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8efe1-51b5-4290-8588-83de5617be7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:07.878591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8efe1-51b5-4290-8588-83de5617be7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:07.880119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8efe1-51b5-4290-8588-83de5617be7b service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:07.890070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1396/2805] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:38:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:38:08.496481 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=78 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:11.918464 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99e84c-1e5a-412b-af85-9e3740991d9b service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:11.920010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ff99e84c-1e5a-412b-af85-9e3740991d9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:11.986509 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99e84c-1e5a-412b-af85-9e3740991d9b service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:11.991150 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:11.992018 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:11.992303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:11.992582 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] PATH_INFO: `/v3/users/6adb938a73894524ac86fa1a264ca38c` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:11.993765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6adb938a73894524ac86fa1a264ca38c)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:11.995706 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:12.199263 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6adb938a73894524ac86fa1a264ca38c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.206831 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6adb938a73894524ac86fa1a264ca38c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.207771 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6adb938a73894524ac86fa1a264ca38c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.208698 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6adb938a73894524ac86fa1a264ca38c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.292258 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8efe1-51b5-4290-8588-83de5617be7b service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:12.298750 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3b4b4569-8020-4cd6-beb5-326358d11329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6adb938a73894524ac86fa1a264ca38c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.303420 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8efe1-51b5-4290-8588-83de5617be7b service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:12.312536 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:12.313430 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:12.313708 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:12.313994 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] PATH_INFO: `/v3/users/3f1e565309324da58709c22047a641f0` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:12.315280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f1e565309324da58709c22047a641f0)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:12.317217 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:12.323549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1410/2806] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:38:11 2022] DELETE /identity/v3/users/6adb938a73894524ac86fa1a264ca38c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:12.327963 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d59e-4982-4ef7-b6b2-b580dcbc21a2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:12.328348 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d59e-4982-4ef7-b6b2-b580dcbc21a2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:12.329214 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913d59e-4982-4ef7-b6b2-b580dcbc21a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:12.348529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-8913d59e-4982-4ef7-b6b2-b580dcbc21a2 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:12.349755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8913d59e-4982-4ef7-b6b2-b580dcbc21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:12.410330 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1411/2807] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:38:12.419942 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d59e-4982-4ef7-b6b2-b580dcbc21a2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:12.421410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-8913d59e-4982-4ef7-b6b2-b580dcbc21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:12.444209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:12.480380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f1e565309324da58709c22047a641f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.481016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f1e565309324da58709c22047a641f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.481816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913d59e-4982-4ef7-b6b2-b580dcbc21a2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:12.487470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8787b6-a45b-472a-9159-1599d5d69720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:12.487470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8787b6-a45b-472a-9159-1599d5d69720 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:12.487470 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8787b6-a45b-472a-9159-1599d5d69720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:12.487825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8787b6-a45b-472a-9159-1599d5d69720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:12.489873 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-aa8787b6-a45b-472a-9159-1599d5d69720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:12.495369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f1e565309324da58709c22047a641f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.496108 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f1e565309324da58709c22047a641f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.504484 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8787b6-a45b-472a-9159-1599d5d69720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:12.505930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8787b6-a45b-472a-9159-1599d5d69720 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:12.517839 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1412/2808] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:38:12.624096 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f1e565309324da58709c22047a641f0'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:12.641794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1397/2809] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:38:12 2022] DELETE /identity/v3/users/3f1e565309324da58709c22047a641f0 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:38:12.647518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549be6f-6126-48aa-9372-a860a9098733 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:12.647812 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549be6f-6126-48aa-9372-a860a9098733 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:12.648164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549be6f-6126-48aa-9372-a860a9098733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:12.666892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2549be6f-6126-48aa-9372-a860a9098733 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:12.667800 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2549be6f-6126-48aa-9372-a860a9098733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:12.749884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1413/2810] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:12.760869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:12.762369 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:12.830861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c984c-b920-49cc-a42d-26145005677f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:12.835270 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd138e-f38d-497b-808c-80f9f0e36812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:12.836021 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd138e-f38d-497b-808c-80f9f0e36812 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:12.836374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd138e-f38d-497b-808c-80f9f0e36812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:12.836722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd138e-f38d-497b-808c-80f9f0e36812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:12.846690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd138e-f38d-497b-808c-80f9f0e36812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:12.848129 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd138e-f38d-497b-808c-80f9f0e36812 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:12.857746 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1398/2811] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:38:12.926192 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549be6f-6126-48aa-9372-a860a9098733 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:12.927691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2549be6f-6126-48aa-9372-a860a9098733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:12.941742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549be6f-6126-48aa-9372-a860a9098733 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:12.946676 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:12.947569 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:12.947854 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:12.948184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] PATH_INFO: `/v3/projects/0280b11c02534bc2b9b81e226e0e39d6` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:12.949691 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0280b11c02534bc2b9b81e226e0e39d6)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:12.951771 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:13.086093 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0280b11c02534bc2b9b81e226e0e39d6', 'request_id': 'req-843e0276-9fe0-4827-80a1-c8d91f60018b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.092711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0280b11c02534bc2b9b81e226e0e39d6', 'request_id': 'req-843e0276-9fe0-4827-80a1-c8d91f60018b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.144783 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0280b11c02534bc2b9b81e226e0e39d6', 'request_id': 'req-843e0276-9fe0-4827-80a1-c8d91f60018b'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.156562 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] The token cache is being invalidate because project 0280b11c02534bc2b9b81e226e0e39d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:38:13.158434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.158434 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-843e0276-9fe0-4827-80a1-c8d91f60018b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.159744 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1414/2812] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:38:12 2022] DELETE /identity/v3/projects/0280b11c02534bc2b9b81e226e0e39d6 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:13.164684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6cb1c-54f0-4bc5-b531-05d93ce9d3a8 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.165047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6cb1c-54f0-4bc5-b531-05d93ce9d3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.165265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6cb1c-54f0-4bc5-b531-05d93ce9d3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.183694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-81e6cb1c-54f0-4bc5-b531-05d93ce9d3a8 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:13.184926 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-81e6cb1c-54f0-4bc5-b531-05d93ce9d3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:13.193766 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:13.247477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1399/2813] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:38:13.257183 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44802f-e231-4d42-a2e8-cfaf16a484c2 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.257476 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44802f-e231-4d42-a2e8-cfaf16a484c2 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.257765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44802f-e231-4d42-a2e8-cfaf16a484c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.277404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-2c44802f-e231-4d42-a2e8-cfaf16a484c2 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:13.278573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2c44802f-e231-4d42-a2e8-cfaf16a484c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:13.287044 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1415/2814] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:13.291291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6cb1c-54f0-4bc5-b531-05d93ce9d3a8 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:13.294557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44802f-e231-4d42-a2e8-cfaf16a484c2 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:13.295923 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2c44802f-e231-4d42-a2e8-cfaf16a484c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:13.303340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6cb1c-54f0-4bc5-b531-05d93ce9d3a8 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:13.309064 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44802f-e231-4d42-a2e8-cfaf16a484c2 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:13.312345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e42ab-412a-4c54-99c9-8a1685ffa055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:13.313611 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e42ab-412a-4c54-99c9-8a1685ffa055 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.314023 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e42ab-412a-4c54-99c9-8a1685ffa055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.314176 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e42ab-412a-4c54-99c9-8a1685ffa055 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.314869 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:13.315015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e42ab-412a-4c54-99c9-8a1685ffa055 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:38:13.315277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e42ab-412a-4c54-99c9-8a1685ffa055 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:13.315894 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.316184 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.316457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] PATH_INFO: `/v3/projects/d59b1bab37224711b576880eca2297be` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.317475 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e42ab-412a-4c54-99c9-8a1685ffa055 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:13.317673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d59b1bab37224711b576880eca2297be)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:13.319611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:13.333085 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1400/2815] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:38:13.337622 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e42ab-412a-4c54-99c9-8a1685ffa055 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:13.348243 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e42ab-412a-4c54-99c9-8a1685ffa055 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:13.353232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171edec-81b7-4a45-ad13-ddfb5fb1d6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:13.353892 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171edec-81b7-4a45-ad13-ddfb5fb1d6ce admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.354181 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171edec-81b7-4a45-ad13-ddfb5fb1d6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.354495 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171edec-81b7-4a45-ad13-ddfb5fb1d6ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.355196 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171edec-81b7-4a45-ad13-ddfb5fb1d6ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:13.357100 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171edec-81b7-4a45-ad13-ddfb5fb1d6ce admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:13.433475 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:13.443410 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd59b1bab37224711b576880eca2297be', 'request_id': 'req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.448898 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd59b1bab37224711b576880eca2297be', 'request_id': 'req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.461968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1401/2816] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:38:13 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:38:13.465974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171edec-81b7-4a45-ad13-ddfb5fb1d6ce admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:13.527867 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd59b1bab37224711b576880eca2297be', 'request_id': 'req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.530316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171edec-81b7-4a45-ad13-ddfb5fb1d6ce admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:13.535989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa021f2-3935-43de-a58c-5aa6c6bef881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:13.536656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa021f2-3935-43de-a58c-5aa6c6bef881 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.536943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa021f2-3935-43de-a58c-5aa6c6bef881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.537227 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa021f2-3935-43de-a58c-5aa6c6bef881 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.537963 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa021f2-3935-43de-a58c-5aa6c6bef881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:13.539888 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa021f2-3935-43de-a58c-5aa6c6bef881 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:13.540181 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] The token cache is being invalidate because project d59b1bab37224711b576880eca2297be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:38:13.540479 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.541271 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.545901 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1416/2817] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:38:13 2022] DELETE /identity/v3/projects/d59b1bab37224711b576880eca2297be => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:38:13.546938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:13.548495 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:13.614492 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1c77c-c9c1-4d98-b9bd-7f52036f7980 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:13.619166 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:13.620066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.620346 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.620617 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] PATH_INFO: `/v3/users/66c247a1a01f4a1eb5910ab627ff07e2` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.621954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66c247a1a01f4a1eb5910ab627ff07e2)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:13.624316 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:13.639232 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1402/2818] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:38:13 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:38:13.640677 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa021f2-3935-43de-a58c-5aa6c6bef881 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:13.642876 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-baa021f2-3935-43de-a58c-5aa6c6bef881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:13.654894 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa021f2-3935-43de-a58c-5aa6c6bef881 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:13.660343 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b580894-ff5f-4bff-b1f5-9a13a6210205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:13.661000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b580894-ff5f-4bff-b1f5-9a13a6210205 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.661293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b580894-ff5f-4bff-b1f5-9a13a6210205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.661559 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b580894-ff5f-4bff-b1f5-9a13a6210205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.662283 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b580894-ff5f-4bff-b1f5-9a13a6210205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:13.663897 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b580894-ff5f-4bff-b1f5-9a13a6210205 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:13.674044 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1403/2819] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:38:13.677905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b580894-ff5f-4bff-b1f5-9a13a6210205 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:13.687579 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b580894-ff5f-4bff-b1f5-9a13a6210205 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:13.692419 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab15eb-7621-4415-97c7-d7d53aebcc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:13.693130 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab15eb-7621-4415-97c7-d7d53aebcc75 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.693330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab15eb-7621-4415-97c7-d7d53aebcc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.693603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab15eb-7621-4415-97c7-d7d53aebcc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.694310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab15eb-7621-4415-97c7-d7d53aebcc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:13.695905 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eab15eb-7621-4415-97c7-d7d53aebcc75 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:13.706985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1404/2820] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:38:13.711013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab15eb-7621-4415-97c7-d7d53aebcc75 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:13.721614 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab15eb-7621-4415-97c7-d7d53aebcc75 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:13.727400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3fca2-fc4b-478f-b71c-7497f72b5435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:13.731148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3fca2-fc4b-478f-b71c-7497f72b5435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.731148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3fca2-fc4b-478f-b71c-7497f72b5435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.731148 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3fca2-fc4b-478f-b71c-7497f72b5435 admin admin] PATH_INFO: `/v3/projects/d3a4bb3cecc4474e85a49a217fa98d9f/users/2273aeca59c04c778683e70665b7efa1/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.745477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3fca2-fc4b-478f-b71c-7497f72b5435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a4bb3cecc4474e85a49a217fa98d9f, user_id=2273aeca59c04c778683e70665b7efa1, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:13.748180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d3fca2-fc4b-478f-b71c-7497f72b5435 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:13.836896 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1405/2821] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:38:13 2022] PUT /identity/v3/projects/d3a4bb3cecc4474e85a49a217fa98d9f/users/2273aeca59c04c778683e70665b7efa1/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:38:13.841851 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae394345-ffe2-4c7d-a9c7-620a28c21933 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:13.842145 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae394345-ffe2-4c7d-a9c7-620a28c21933 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:13.842460 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae394345-ffe2-4c7d-a9c7-620a28c21933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:13.860365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-ae394345-ffe2-4c7d-a9c7-620a28c21933 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:13.861594 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ae394345-ffe2-4c7d-a9c7-620a28c21933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:13.867202 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66c247a1a01f4a1eb5910ab627ff07e2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.874888 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66c247a1a01f4a1eb5910ab627ff07e2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.876915 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66c247a1a01f4a1eb5910ab627ff07e2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.877556 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66c247a1a01f4a1eb5910ab627ff07e2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.880484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:13.934277 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1406/2822] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:38:13.944806 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae394345-ffe2-4c7d-a9c7-620a28c21933 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:13.946164 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ae394345-ffe2-4c7d-a9c7-620a28c21933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:13.950503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66c247a1a01f4a1eb5910ab627ff07e2'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:13.969921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1417/2823] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:38:13 2022] DELETE /identity/v3/users/66c247a1a01f4a1eb5910ab627ff07e2 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:14.006367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae394345-ffe2-4c7d-a9c7-620a28c21933 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:14.056799 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f260d-614f-4fd8-9eec-a0efae14446c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:14.057458 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f260d-614f-4fd8-9eec-a0efae14446c service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:14.057742 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f260d-614f-4fd8-9eec-a0efae14446c service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:14.058020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f260d-614f-4fd8-9eec-a0efae14446c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:14.118591 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f260d-614f-4fd8-9eec-a0efae14446c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:14.120022 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f260d-614f-4fd8-9eec-a0efae14446c service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:14.130321 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1407/2824] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:38:14.414793 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:14.416082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:14.454503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0d1f-192f-439e-abc4-1e0c55b922f6 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:14.462427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:14.463991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:14.464494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:14.464991 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] PATH_INFO: `/v3/projects/e5021de967244c8f9baf3155cc93f1e3` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:14.467155 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5021de967244c8f9baf3155cc93f1e3)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:14.471645 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:14.620337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:14.633190 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5021de967244c8f9baf3155cc93f1e3', 'request_id': 'req-b10e7e81-973d-4c17-8c05-a8c2d76764ca'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:14.637079 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5021de967244c8f9baf3155cc93f1e3', 'request_id': 'req-b10e7e81-973d-4c17-8c05-a8c2d76764ca'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:14.710756 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5021de967244c8f9baf3155cc93f1e3', 'request_id': 'req-b10e7e81-973d-4c17-8c05-a8c2d76764ca'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:14.725501 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] The token cache is being invalidate because project e5021de967244c8f9baf3155cc93f1e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:38:14.725843 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:14.726466 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:14.728057 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1418/2825] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:38:14 2022] DELETE /identity/v3/projects/e5021de967244c8f9baf3155cc93f1e3 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:14.732181 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f260d-614f-4fd8-9eec-a0efae14446c service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:14.743114 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f260d-614f-4fd8-9eec-a0efae14446c service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:14.770380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:14.800285 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:14.801079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:14.801360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:14.801633 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] PATH_INFO: `/v3/users/547b9a48a7c241e8b9b9e755211aef90` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:14.802881 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=547b9a48a7c241e8b9b9e755211aef90)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:14.804796 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:15.050464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547b9a48a7c241e8b9b9e755211aef90'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.051126 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547b9a48a7c241e8b9b9e755211aef90'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.065911 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547b9a48a7c241e8b9b9e755211aef90'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.066686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547b9a48a7c241e8b9b9e755211aef90'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.104697 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-5882fbb5-296e-4292-a08d-df95a3122743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547b9a48a7c241e8b9b9e755211aef90'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.119140 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1408/2826] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:38:14 2022] DELETE /identity/v3/users/547b9a48a7c241e8b9b9e755211aef90 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:15.570641 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:15.572416 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:15.588822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7e81-973d-4c17-8c05-a8c2d76764ca admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:15.673352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:15.674262 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:15.674607 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:15.674906 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] PATH_INFO: `/v3/projects/4e65b9036a8f4296b688c119e37c4d8e` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:15.676148 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e65b9036a8f4296b688c119e37c4d8e)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:15.678170 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:15.804549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e65b9036a8f4296b688c119e37c4d8e', 'request_id': 'req-f9569743-ea7f-4105-8043-122c9f372516'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.810515 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e65b9036a8f4296b688c119e37c4d8e', 'request_id': 'req-f9569743-ea7f-4105-8043-122c9f372516'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.887494 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e65b9036a8f4296b688c119e37c4d8e', 'request_id': 'req-f9569743-ea7f-4105-8043-122c9f372516'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.901496 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] The token cache is being invalidate because project 4e65b9036a8f4296b688c119e37c4d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:38:15.901787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.902452 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-f9569743-ea7f-4105-8043-122c9f372516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:15.904228 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1419/2827] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:38:15 2022] DELETE /identity/v3/projects/4e65b9036a8f4296b688c119e37c4d8e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:15.913175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025c0f9-5131-4639-b81e-588cd1729725 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:15.914391 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025c0f9-5131-4639-b81e-588cd1729725 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:15.914706 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025c0f9-5131-4639-b81e-588cd1729725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:15.933666 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-e025c0f9-5131-4639-b81e-588cd1729725 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:15.935189 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-e025c0f9-5131-4639-b81e-588cd1729725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:16.006193 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1409/2828] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:16.015283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf63c11-bd81-4a49-bc01-db25902d1abd None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:16.015646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf63c11-bd81-4a49-bc01-db25902d1abd None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:16.016544 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf63c11-bd81-4a49-bc01-db25902d1abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:16.038752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-7cf63c11-bd81-4a49-bc01-db25902d1abd None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:16.040048 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-7cf63c11-bd81-4a49-bc01-db25902d1abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:16.048594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1420/2829] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:16.058731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025c0f9-5131-4639-b81e-588cd1729725 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:16.068484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025c0f9-5131-4639-b81e-588cd1729725 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:16.073093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c63fc-e4c0-4cc0-bf42-65bc537bb71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:16.073779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c63fc-e4c0-4cc0-bf42-65bc537bb71a admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:16.074064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c63fc-e4c0-4cc0-bf42-65bc537bb71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:16.074477 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c63fc-e4c0-4cc0-bf42-65bc537bb71a admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:16.075105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c63fc-e4c0-4cc0-bf42-65bc537bb71a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:38:16.075410 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c63fc-e4c0-4cc0-bf42-65bc537bb71a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:16.077000 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c63fc-e4c0-4cc0-bf42-65bc537bb71a admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:16.090052 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1410/2830] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:38:16.095029 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf63c11-bd81-4a49-bc01-db25902d1abd None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:16.104616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf63c11-bd81-4a49-bc01-db25902d1abd None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:16.109650 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2870a-cd2e-4ce0-8135-2fc28c23d392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:16.110686 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2870a-cd2e-4ce0-8135-2fc28c23d392 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:16.110908 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2870a-cd2e-4ce0-8135-2fc28c23d392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:16.111152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2870a-cd2e-4ce0-8135-2fc28c23d392 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:16.111830 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2870a-cd2e-4ce0-8135-2fc28c23d392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:16.114047 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2870a-cd2e-4ce0-8135-2fc28c23d392 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:16.239709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1421/2831] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:38:16 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:38:16.241337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c63fc-e4c0-4cc0-bf42-65bc537bb71a admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:16.252469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c63fc-e4c0-4cc0-bf42-65bc537bb71a admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:16.302759 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:16.331444 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209bc81-0551-4600-88da-b22367d646cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:16.332652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209bc81-0551-4600-88da-b22367d646cf admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:16.332652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209bc81-0551-4600-88da-b22367d646cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:16.332652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209bc81-0551-4600-88da-b22367d646cf admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:16.333435 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209bc81-0551-4600-88da-b22367d646cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:16.339679 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209bc81-0551-4600-88da-b22367d646cf admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:16.501743 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1411/2832] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:38:16 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:38:16.506102 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2870a-cd2e-4ce0-8135-2fc28c23d392 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:16.517654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2870a-cd2e-4ce0-8135-2fc28c23d392 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:16.522251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46e656-68cb-4272-b91f-c3b2226b08da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:16.522954 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46e656-68cb-4272-b91f-c3b2226b08da admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:16.523237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46e656-68cb-4272-b91f-c3b2226b08da admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:16.523513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46e656-68cb-4272-b91f-c3b2226b08da admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:16.524230 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46e656-68cb-4272-b91f-c3b2226b08da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:16.525816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46e656-68cb-4272-b91f-c3b2226b08da admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:16.536708 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1422/2833] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:38:16.542877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209bc81-0551-4600-88da-b22367d646cf admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:16.554570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209bc81-0551-4600-88da-b22367d646cf admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:16.560153 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90911c63-7842-4412-b7c3-810ed11cd07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:16.560833 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90911c63-7842-4412-b7c3-810ed11cd07c admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:16.561125 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90911c63-7842-4412-b7c3-810ed11cd07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:16.561413 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90911c63-7842-4412-b7c3-810ed11cd07c admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:16.565432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90911c63-7842-4412-b7c3-810ed11cd07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:16.565432 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90911c63-7842-4412-b7c3-810ed11cd07c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:16.576095 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1412/2834] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 12:38:16.580660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46e656-68cb-4272-b91f-c3b2226b08da admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:16.591164 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46e656-68cb-4272-b91f-c3b2226b08da admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:16.600008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59d0d2-0824-45de-8162-eabdee20f67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:16.600008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59d0d2-0824-45de-8162-eabdee20f67f admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:16.600008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59d0d2-0824-45de-8162-eabdee20f67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:16.600008 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59d0d2-0824-45de-8162-eabdee20f67f admin admin] PATH_INFO: `/v3/projects/9e0a9d97b8db4692bd5b1d85b434fab0/users/8d875bc37f9b45aa9783abcb34e6d34f/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:16.612471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:16.616135 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59d0d2-0824-45de-8162-eabdee20f67f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0a9d97b8db4692bd5b1d85b434fab0, user_id=8d875bc37f9b45aa9783abcb34e6d34f, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:16.618975 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b59d0d2-0824-45de-8162-eabdee20f67f admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:16.752587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1423/2835] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:38:16 2022] PUT /identity/v3/projects/9e0a9d97b8db4692bd5b1d85b434fab0/users/8d875bc37f9b45aa9783abcb34e6d34f/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:16.764815 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b31bc-4aa0-4202-be05-34e009f3d10e None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:16.765247 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b31bc-4aa0-4202-be05-34e009f3d10e None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:16.765529 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b31bc-4aa0-4202-be05-34e009f3d10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:16.794161 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-cc3b31bc-4aa0-4202-be05-34e009f3d10e None None] MFA Rules not processed for user `8d875bc37f9b45aa9783abcb34e6d34f`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:16.795464 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-cc3b31bc-4aa0-4202-be05-34e009f3d10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:16.875252 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1413/2836] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:16.876740 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59d0d2-0824-45de-8162-eabdee20f67f admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:16.949451 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b59d0d2-0824-45de-8162-eabdee20f67f admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:16.954917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aaa7d1-cc62-437f-a786-ac326fde4519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:16.955683 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaa7d1-cc62-437f-a786-ac326fde4519 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:16.955971 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaa7d1-cc62-437f-a786-ac326fde4519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:16.956248 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaa7d1-cc62-437f-a786-ac326fde4519 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:16.957037 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aaa7d1-cc62-437f-a786-ac326fde4519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:16.959082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aaa7d1-cc62-437f-a786-ac326fde4519 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:17.057207 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1424/2837] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:38:16 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 12:38:17.061807 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b31bc-4aa0-4202-be05-34e009f3d10e None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:17.074705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b31bc-4aa0-4202-be05-34e009f3d10e None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:17.149337 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e2b6-3bc4-42bd-bd2d-671bf98f66ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:17.150056 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e2b6-3bc4-42bd-bd2d-671bf98f66ae admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:17.150340 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e2b6-3bc4-42bd-bd2d-671bf98f66ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:17.150675 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e2b6-3bc4-42bd-bd2d-671bf98f66ae admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:17.152877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79e2b6-3bc4-42bd-bd2d-671bf98f66ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:17.154779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79e2b6-3bc4-42bd-bd2d-671bf98f66ae admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:17.261514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1414/2838] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:38:17 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:38:17.266375 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aaa7d1-cc62-437f-a786-ac326fde4519 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:17.279951 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aaa7d1-cc62-437f-a786-ac326fde4519 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:17.287964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d7c07-175f-4023-990c-4ed9174f31af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:17.288940 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d7c07-175f-4023-990c-4ed9174f31af admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:17.289229 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d7c07-175f-4023-990c-4ed9174f31af admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:17.289511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d7c07-175f-4023-990c-4ed9174f31af admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:17.290274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d7c07-175f-4023-990c-4ed9174f31af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:17.292060 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d7c07-175f-4023-990c-4ed9174f31af admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:17.307027 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1425/2839] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:38:17.314215 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e2b6-3bc4-42bd-bd2d-671bf98f66ae admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:17.325542 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e2b6-3bc4-42bd-bd2d-671bf98f66ae admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:17.331034 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031dbe8-69ef-4d7b-928a-434aebbc1039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:17.331998 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031dbe8-69ef-4d7b-928a-434aebbc1039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:17.333027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031dbe8-69ef-4d7b-928a-434aebbc1039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:17.333027 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031dbe8-69ef-4d7b-928a-434aebbc1039 admin admin] PATH_INFO: `/v3/projects/5d9ab467ba4648d19426541f6bf13884/users/2e46cb7500d34c44bbbd0d26e9c2032b/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:17.351732 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031dbe8-69ef-4d7b-928a-434aebbc1039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9ab467ba4648d19426541f6bf13884, user_id=2e46cb7500d34c44bbbd0d26e9c2032b, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:17.354770 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031dbe8-69ef-4d7b-928a-434aebbc1039 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:17.428330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1415/2840] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:38:17 2022] PUT /identity/v3/projects/5d9ab467ba4648d19426541f6bf13884/users/2e46cb7500d34c44bbbd0d26e9c2032b/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:17.431928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d7c07-175f-4023-990c-4ed9174f31af admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:17.442926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d7c07-175f-4023-990c-4ed9174f31af admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:17.523536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ac59f-0f71-4213-a63d-906cd1a5702e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:17.524297 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ac59f-0f71-4213-a63d-906cd1a5702e admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:17.524711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ac59f-0f71-4213-a63d-906cd1a5702e admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:17.525082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ac59f-0f71-4213-a63d-906cd1a5702e admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:17.525897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ac59f-0f71-4213-a63d-906cd1a5702e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:17.527768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ac59f-0f71-4213-a63d-906cd1a5702e admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:17.543924 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1426/2841] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:38:17.548832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031dbe8-69ef-4d7b-928a-434aebbc1039 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:17.585079 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031dbe8-69ef-4d7b-928a-434aebbc1039 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:17.597424 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6215-0617-45b3-9e13-72f08e84ab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:17.600701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6215-0617-45b3-9e13-72f08e84ab44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:17.601001 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6215-0617-45b3-9e13-72f08e84ab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:17.601341 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6215-0617-45b3-9e13-72f08e84ab44 admin admin] PATH_INFO: `/v3/projects/5d9ab467ba4648d19426541f6bf13884/users/2e46cb7500d34c44bbbd0d26e9c2032b/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:17.611440 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb6215-0617-45b3-9e13-72f08e84ab44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9ab467ba4648d19426541f6bf13884, user_id=2e46cb7500d34c44bbbd0d26e9c2032b, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:17.614944 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb6215-0617-45b3-9e13-72f08e84ab44 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:17.624686 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:17.642608 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1416/2842] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:38:17 2022] PUT /identity/v3/projects/5d9ab467ba4648d19426541f6bf13884/users/2e46cb7500d34c44bbbd0d26e9c2032b/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:17.713351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096c9bf-ec54-4ba7-9a1d-e8772ca3b52d None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:17.713926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096c9bf-ec54-4ba7-9a1d-e8772ca3b52d None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:17.713926 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096c9bf-ec54-4ba7-9a1d-e8772ca3b52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:17.744725 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b096c9bf-ec54-4ba7-9a1d-e8772ca3b52d None None] MFA Rules not processed for user `2e46cb7500d34c44bbbd0d26e9c2032b`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:17.746443 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b096c9bf-ec54-4ba7-9a1d-e8772ca3b52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:17.815749 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:17.839981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1427/2843] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:17.854958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6215-0617-45b3-9e13-72f08e84ab44 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:17.859702 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-8ddb6215-0617-45b3-9e13-72f08e84ab44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:17.945506 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6215-0617-45b3-9e13-72f08e84ab44 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:17.949399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb25d8-f7fe-4018-a9af-5dff10f64383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:17.950005 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb25d8-f7fe-4018-a9af-5dff10f64383 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:17.950684 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb25d8-f7fe-4018-a9af-5dff10f64383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:17.950985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb25d8-f7fe-4018-a9af-5dff10f64383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:18.038603 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb25d8-f7fe-4018-a9af-5dff10f64383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:18.040078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb25d8-f7fe-4018-a9af-5dff10f64383 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:18.052367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1417/2844] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:38:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:38:19.764152 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b70e07-d3c4-47b8-8cf0-72147c77cf23 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:19.764752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b70e07-d3c4-47b8-8cf0-72147c77cf23 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:19.764752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b70e07-d3c4-47b8-8cf0-72147c77cf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:19.797274 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-28b70e07-d3c4-47b8-8cf0-72147c77cf23 None None] MFA Rules not processed for user `2273aeca59c04c778683e70665b7efa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:19.798715 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-28b70e07-d3c4-47b8-8cf0-72147c77cf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:19.912689 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1428/2845] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:19.926443 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb25d8-f7fe-4018-a9af-5dff10f64383 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:19.928236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-57eb25d8-f7fe-4018-a9af-5dff10f64383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:20.022683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb25d8-f7fe-4018-a9af-5dff10f64383 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:20.031400 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf8b7-37aa-44b4-851b-6cf69de183af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:20.032115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf8b7-37aa-44b4-851b-6cf69de183af admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:20.032598 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf8b7-37aa-44b4-851b-6cf69de183af admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:20.033069 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf8b7-37aa-44b4-851b-6cf69de183af admin admin] PATH_INFO: `/v3/projects` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:20.033817 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdf8b7-37aa-44b4-851b-6cf69de183af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:20.035832 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdf8b7-37aa-44b4-851b-6cf69de183af admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:20.143105 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1418/2846] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:38:19 2022] POST /identity/v3/projects => generated 371 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:38:20.147511 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b70e07-d3c4-47b8-8cf0-72147c77cf23 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:20.163337 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b70e07-d3c4-47b8-8cf0-72147c77cf23 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:20.250900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383317d-cdab-4246-a4fa-df7627691ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:20.251759 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383317d-cdab-4246-a4fa-df7627691ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:20.252051 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383317d-cdab-4246-a4fa-df7627691ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:20.252371 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383317d-cdab-4246-a4fa-df7627691ce8 admin admin] PATH_INFO: `/v3/users` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:20.253179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383317d-cdab-4246-a4fa-df7627691ce8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:20.255132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383317d-cdab-4246-a4fa-df7627691ce8 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:20.346536 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1429/2847] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:38:20 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:38:20.350942 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf8b7-37aa-44b4-851b-6cf69de183af admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:20.401811 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf8b7-37aa-44b4-851b-6cf69de183af admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:20.408889 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87046f6-0898-4e46-b844-84ef0070b796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:20.409564 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87046f6-0898-4e46-b844-84ef0070b796 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:20.409845 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87046f6-0898-4e46-b844-84ef0070b796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:20.410118 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87046f6-0898-4e46-b844-84ef0070b796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:20.410920 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87046f6-0898-4e46-b844-84ef0070b796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:20.412615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87046f6-0898-4e46-b844-84ef0070b796 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:20.429292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1419/2848] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:38:20.431537 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383317d-cdab-4246-a4fa-df7627691ce8 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:20.443360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383317d-cdab-4246-a4fa-df7627691ce8 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:20.448550 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483be501-066e-4967-8ba9-cc3dec7b8f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:20.449473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483be501-066e-4967-8ba9-cc3dec7b8f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:20.449765 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483be501-066e-4967-8ba9-cc3dec7b8f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:20.450050 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483be501-066e-4967-8ba9-cc3dec7b8f10 admin admin] PATH_INFO: `/v3/projects/99d9ea0ac1794cee8e7651632c0e88dd/users/7f6f34a427aa4f348afb98a6b5987ef5/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:20.467736 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483be501-066e-4967-8ba9-cc3dec7b8f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d9ea0ac1794cee8e7651632c0e88dd, user_id=7f6f34a427aa4f348afb98a6b5987ef5, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:20.473528 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483be501-066e-4967-8ba9-cc3dec7b8f10 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:20.574850 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1430/2849] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:38:20 2022] PUT /identity/v3/projects/99d9ea0ac1794cee8e7651632c0e88dd/users/7f6f34a427aa4f348afb98a6b5987ef5/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:20.580526 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87046f6-0898-4e46-b844-84ef0070b796 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:20.595958 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87046f6-0898-4e46-b844-84ef0070b796 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:20.637662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:20.691134 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86221614-4f63-48fe-9dd0-e45b145e8b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:20.691916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86221614-4f63-48fe-9dd0-e45b145e8b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:20.692229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86221614-4f63-48fe-9dd0-e45b145e8b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:20.692484 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86221614-4f63-48fe-9dd0-e45b145e8b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:20.693229 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86221614-4f63-48fe-9dd0-e45b145e8b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:20.694919 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86221614-4f63-48fe-9dd0-e45b145e8b39 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:20.710538 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1420/2850] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:38:20.712548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483be501-066e-4967-8ba9-cc3dec7b8f10 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:20.724626 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483be501-066e-4967-8ba9-cc3dec7b8f10 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:20.729897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce27fd2-b883-4d02-8d78-f00ec76c2b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:20.731017 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce27fd2-b883-4d02-8d78-f00ec76c2b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:20.731419 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce27fd2-b883-4d02-8d78-f00ec76c2b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:20.731825 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce27fd2-b883-4d02-8d78-f00ec76c2b4a admin admin] PATH_INFO: `/v3/projects/99d9ea0ac1794cee8e7651632c0e88dd/users/7f6f34a427aa4f348afb98a6b5987ef5/roles/bd295cb5d1194b469beff0381c6b23af` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:20.735427 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce27fd2-b883-4d02-8d78-f00ec76c2b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d9ea0ac1794cee8e7651632c0e88dd, user_id=7f6f34a427aa4f348afb98a6b5987ef5, role_id=bd295cb5d1194b469beff0381c6b23af)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:20.738840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce27fd2-b883-4d02-8d78-f00ec76c2b4a admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:20.755594 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1431/2851] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:38:20 2022] PUT /identity/v3/projects/99d9ea0ac1794cee8e7651632c0e88dd/users/7f6f34a427aa4f348afb98a6b5987ef5/roles/bd295cb5d1194b469beff0381c6b23af => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 12:38:25.108345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86221614-4f63-48fe-9dd0-e45b145e8b39 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:25.177187 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86221614-4f63-48fe-9dd0-e45b145e8b39 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:25.241223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6e81a-6b95-4335-a8b6-d03b08d8d4c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:25.242103 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6e81a-6b95-4335-a8b6-d03b08d8d4c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:25.242489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6e81a-6b95-4335-a8b6-d03b08d8d4c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:25.243388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6e81a-6b95-4335-a8b6-d03b08d8d4c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:25.328862 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6e81a-6b95-4335-a8b6-d03b08d8d4c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:25.330386 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6e81a-6b95-4335-a8b6-d03b08d8d4c2 service cinder] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:25.341064 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1421/2852] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:38:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:38:27.418110 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ebc45-57b0-44ac-97bd-a13fff433bf1 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:27.418664 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ebc45-57b0-44ac-97bd-a13fff433bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:27.418781 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ebc45-57b0-44ac-97bd-a13fff433bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:27.459351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-ed5ebc45-57b0-44ac-97bd-a13fff433bf1 None None] MFA Rules not processed for user `7f6f34a427aa4f348afb98a6b5987ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:27.468554 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ed5ebc45-57b0-44ac-97bd-a13fff433bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:27.491032 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:27.568033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1432/2853] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 12:38:27.589954 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6e81a-6b95-4335-a8b6-d03b08d8d4c2 service cinder] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:27.592074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-51b6e81a-6b95-4335-a8b6-d03b08d8d4c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:27.627989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:27.683869 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6e81a-6b95-4335-a8b6-d03b08d8d4c2 service cinder] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:27.690947 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4c4e9-a361-4985-8834-88bb90cb9e19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:27.691615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4c4e9-a361-4985-8834-88bb90cb9e19 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:27.691898 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4c4e9-a361-4985-8834-88bb90cb9e19 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:27.692171 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4c4e9-a361-4985-8834-88bb90cb9e19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:27.758584 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4c4e9-a361-4985-8834-88bb90cb9e19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:27.762098 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4c4e9-a361-4985-8834-88bb90cb9e19 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:27.782701 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1422/2854] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:38:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:38:30.962010 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0e296-2d88-48eb-b640-ef55183b2878 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:30.962652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0e296-2d88-48eb-b640-ef55183b2878 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:30.962652 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0e296-2d88-48eb-b640-ef55183b2878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:30.983768 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-12e0e296-2d88-48eb-b640-ef55183b2878 None None] MFA Rules not processed for user `2273aeca59c04c778683e70665b7efa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:30.986046 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-12e0e296-2d88-48eb-b640-ef55183b2878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:30.993555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1433/2855] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:31.623552 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4c4e9-a361-4985-8834-88bb90cb9e19 service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:31.722837 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4c4e9-a361-4985-8834-88bb90cb9e19 service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:31.739078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:31.740801 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:31.741404 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:31.742062 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] PATH_INFO: `/v3/users/8d875bc37f9b45aa9783abcb34e6d34f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:31.754006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d875bc37f9b45aa9783abcb34e6d34f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:31.754006 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:32.035214 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d875bc37f9b45aa9783abcb34e6d34f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:32.046077 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d875bc37f9b45aa9783abcb34e6d34f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:32.066556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d875bc37f9b45aa9783abcb34e6d34f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:32.067394 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d875bc37f9b45aa9783abcb34e6d34f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:32.069228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:32.298165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d875bc37f9b45aa9783abcb34e6d34f'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:32.302981 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1423/2856] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:38:31 2022] DELETE /identity/v3/users/8d875bc37f9b45aa9783abcb34e6d34f => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:32.307478 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0fbd5-3632-4911-b980-1ca46d2e0484 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:32.307958 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0fbd5-3632-4911-b980-1ca46d2e0484 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:32.308082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0fbd5-3632-4911-b980-1ca46d2e0484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:32.329399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-b8e0fbd5-3632-4911-b980-1ca46d2e0484 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:32.336987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b8e0fbd5-3632-4911-b980-1ca46d2e0484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:32.414256 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1434/2857] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:32.422908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:32.426345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:32.505509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8602a-5a82-48fc-a825-92c240f30cf5 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:32.510652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf3d74-5e83-4071-92ff-92a15dc7272a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:32.511431 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf3d74-5e83-4071-92ff-92a15dc7272a service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:32.511786 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf3d74-5e83-4071-92ff-92a15dc7272a service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:32.512180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf3d74-5e83-4071-92ff-92a15dc7272a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:32.524754 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf3d74-5e83-4071-92ff-92a15dc7272a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:32.526323 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf3d74-5e83-4071-92ff-92a15dc7272a service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:32.542389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1424/2858] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:38:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 12:38:33.061122 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0fbd5-3632-4911-b980-1ca46d2e0484 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:33.063761 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-b8e0fbd5-3632-4911-b980-1ca46d2e0484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:33.080673 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0fbd5-3632-4911-b980-1ca46d2e0484 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:33.094962 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:33.096146 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:33.096601 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:33.097281 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] PATH_INFO: `/v3/projects/9e0a9d97b8db4692bd5b1d85b434fab0` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:33.103327 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0a9d97b8db4692bd5b1d85b434fab0)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:33.105331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:33.220428 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0a9d97b8db4692bd5b1d85b434fab0', 'request_id': 'req-3d29d2ee-ec07-4f10-9701-2fd85224d17c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.226525 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0a9d97b8db4692bd5b1d85b434fab0', 'request_id': 'req-3d29d2ee-ec07-4f10-9701-2fd85224d17c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.310011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0a9d97b8db4692bd5b1d85b434fab0', 'request_id': 'req-3d29d2ee-ec07-4f10-9701-2fd85224d17c'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.322266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] The token cache is being invalidate because project 9e0a9d97b8db4692bd5b1d85b434fab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:38:33.322585 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.323238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.324844 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1435/2859] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:38:33 2022] DELETE /identity/v3/projects/9e0a9d97b8db4692bd5b1d85b434fab0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:33.329578 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf3d74-5e83-4071-92ff-92a15dc7272a service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:33.340700 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf3d74-5e83-4071-92ff-92a15dc7272a service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:33.408316 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:33.409287 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:33.409571 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:33.409846 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] PATH_INFO: `/v3/users/2e46cb7500d34c44bbbd0d26e9c2032b` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:33.411174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e46cb7500d34c44bbbd0d26e9c2032b)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:33.413099 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:33.606315 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e46cb7500d34c44bbbd0d26e9c2032b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.607115 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e46cb7500d34c44bbbd0d26e9c2032b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.628007 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e46cb7500d34c44bbbd0d26e9c2032b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.628830 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e46cb7500d34c44bbbd0d26e9c2032b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.698365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-9f02086a-ac0f-4498-a699-5a7d9fa3ed25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e46cb7500d34c44bbbd0d26e9c2032b'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:33.712789 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1425/2860] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:38:33 2022] DELETE /identity/v3/users/2e46cb7500d34c44bbbd0d26e9c2032b => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:34.214457 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:34.221513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:34.229023 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29d2ee-ec07-4f10-9701-2fd85224d17c admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:34.259446 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:34.287773 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:34.288644 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:34.288929 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:34.289205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] PATH_INFO: `/v3/projects/5d9ab467ba4648d19426541f6bf13884` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:34.290400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9ab467ba4648d19426541f6bf13884)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:34.292344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:34.395785 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9ab467ba4648d19426541f6bf13884', 'request_id': 'req-2270c145-09ee-4997-a114-b6425f7b99d7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:34.402213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9ab467ba4648d19426541f6bf13884', 'request_id': 'req-2270c145-09ee-4997-a114-b6425f7b99d7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:34.492503 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9ab467ba4648d19426541f6bf13884', 'request_id': 'req-2270c145-09ee-4997-a114-b6425f7b99d7'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:34.505549 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] The token cache is being invalidate because project 5d9ab467ba4648d19426541f6bf13884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:38:34.505835 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:34.506498 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2270c145-09ee-4997-a114-b6425f7b99d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:38:34.508022 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1436/2861] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:38:34 2022] DELETE /identity/v3/projects/5d9ab467ba4648d19426541f6bf13884 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:34.514109 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc21330-aa5d-4f63-adeb-d1c6893245e0 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:34.514615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc21330-aa5d-4f63-adeb-d1c6893245e0 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:34.514739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc21330-aa5d-4f63-adeb-d1c6893245e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:34.522280 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:34.534755 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-6cc21330-aa5d-4f63-adeb-d1c6893245e0 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:34.535975 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6cc21330-aa5d-4f63-adeb-d1c6893245e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:34.605107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1426/2862] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:34.614200 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccabcf9-33ef-4b50-b8d6-ad8860b9d838 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:34.614654 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccabcf9-33ef-4b50-b8d6-ad8860b9d838 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:34.614817 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccabcf9-33ef-4b50-b8d6-ad8860b9d838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:34.636606 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-6ccabcf9-33ef-4b50-b8d6-ad8860b9d838 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:34.637828 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-6ccabcf9-33ef-4b50-b8d6-ad8860b9d838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:34.646627 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1437/2863] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:34.651326 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc21330-aa5d-4f63-adeb-d1c6893245e0 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:34.661514 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc21330-aa5d-4f63-adeb-d1c6893245e0 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:34.666291 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2f856-6fba-4af2-a249-571dda59d1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:34.667047 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2f856-6fba-4af2-a249-571dda59d1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:34.667330 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2f856-6fba-4af2-a249-571dda59d1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:34.667607 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2f856-6fba-4af2-a249-571dda59d1d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:34.668345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2f856-6fba-4af2-a249-571dda59d1d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 12:38:34.668652 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2f856-6fba-4af2-a249-571dda59d1d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:34.670253 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2f856-6fba-4af2-a249-571dda59d1d2 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:34.682739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1427/2864] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 19 12:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 12:38:34.687927 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccabcf9-33ef-4b50-b8d6-ad8860b9d838 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:34.697354 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccabcf9-33ef-4b50-b8d6-ad8860b9d838 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:34.702118 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff7cb4-61c2-4ff2-9818-0e28f0c49e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:34.702987 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff7cb4-61c2-4ff2-9818-0e28f0c49e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:34.703270 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff7cb4-61c2-4ff2-9818-0e28f0c49e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:34.703547 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff7cb4-61c2-4ff2-9818-0e28f0c49e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:34.704233 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff7cb4-61c2-4ff2-9818-0e28f0c49e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:34.706125 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff7cb4-61c2-4ff2-9818-0e28f0c49e84 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:34.798383 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1438/2865] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 19 12:38:34 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:38:34.802465 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2f856-6fba-4af2-a249-571dda59d1d2 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:34.813570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2f856-6fba-4af2-a249-571dda59d1d2 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:34.878509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa565a-61b7-42f7-908f-58f230341b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:34.879175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa565a-61b7-42f7-908f-58f230341b93 admin admin] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:34.879457 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa565a-61b7-42f7-908f-58f230341b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:34.879731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa565a-61b7-42f7-908f-58f230341b93 admin admin] PATH_INFO: `/v3/users` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:34.880426 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa565a-61b7-42f7-908f-58f230341b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:34.882365 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa565a-61b7-42f7-908f-58f230341b93 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:34.987528 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1428/2866] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 19 12:38:34 2022] POST /identity/v3/users => generated 282 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 12:38:34.992258 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff7cb4-61c2-4ff2-9818-0e28f0c49e84 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:35.004822 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff7cb4-61c2-4ff2-9818-0e28f0c49e84 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:35.010616 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1fc02-9d15-458c-a7cd-1b292567381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:35.011646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1fc02-9d15-458c-a7cd-1b292567381d admin admin] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:35.012610 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1fc02-9d15-458c-a7cd-1b292567381d admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:35.012865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1fc02-9d15-458c-a7cd-1b292567381d admin admin] PATH_INFO: `/v3/roles` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:35.013702 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1fc02-9d15-458c-a7cd-1b292567381d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:35.015396 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1fc02-9d15-458c-a7cd-1b292567381d admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:35.028400 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1439/2867] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:38:35.033093 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa565a-61b7-42f7-908f-58f230341b93 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:35.043319 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa565a-61b7-42f7-908f-58f230341b93 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:35.048730 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35351ea-fdbf-455c-aae9-10b1e8151b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:35.049388 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35351ea-fdbf-455c-aae9-10b1e8151b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:35.049667 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35351ea-fdbf-455c-aae9-10b1e8151b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:35.049943 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35351ea-fdbf-455c-aae9-10b1e8151b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:35.050696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35351ea-fdbf-455c-aae9-10b1e8151b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:35.052345 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35351ea-fdbf-455c-aae9-10b1e8151b4d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:35.063070 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1429/2868] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 19 12:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 12:38:35.067450 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1fc02-9d15-458c-a7cd-1b292567381d admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:35.078995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1fc02-9d15-458c-a7cd-1b292567381d admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:35.083752 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ec7ce-2162-4422-9571-3f377b840f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:35.084629 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ec7ce-2162-4422-9571-3f377b840f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:35.084902 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ec7ce-2162-4422-9571-3f377b840f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:35.085174 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ec7ce-2162-4422-9571-3f377b840f1b admin admin] PATH_INFO: `/v3/projects/ecbc0d593e974432a4aceaf6044460bd/users/f3fef41cb8b54f0bbc62c57995475fb8/roles/60e46f07f6ab4fdd90ec5d32066e0faa` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:35.101672 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ec7ce-2162-4422-9571-3f377b840f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbc0d593e974432a4aceaf6044460bd, user_id=f3fef41cb8b54f0bbc62c57995475fb8, role_id=60e46f07f6ab4fdd90ec5d32066e0faa)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:35.104395 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ec7ce-2162-4422-9571-3f377b840f1b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:35.162471 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1440/2869] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 19 12:38:35 2022] PUT /identity/v3/projects/ecbc0d593e974432a4aceaf6044460bd/users/f3fef41cb8b54f0bbc62c57995475fb8/roles/60e46f07f6ab4fdd90ec5d32066e0faa => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:38:35.175368 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777421d8-2dd8-4b4a-8c4c-5436637cbe5a None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:35.175715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777421d8-2dd8-4b4a-8c4c-5436637cbe5a None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:35.175999 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777421d8-2dd8-4b4a-8c4c-5436637cbe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:35.207413 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-777421d8-2dd8-4b4a-8c4c-5436637cbe5a None None] MFA Rules not processed for user `f3fef41cb8b54f0bbc62c57995475fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:38:35.208558 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-777421d8-2dd8-4b4a-8c4c-5436637cbe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:35.269602 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:35.273223 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1430/2870] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:38:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:38:35.284976 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ec7ce-2162-4422-9571-3f377b840f1b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:38:35.286731 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-bd8ec7ce-2162-4422-9571-3f377b840f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:38:35.346721 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ec7ce-2162-4422-9571-3f377b840f1b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:38:35.350315 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47571ce9-5050-4c8f-990b-fa4824ff57b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:38:35.351115 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47571ce9-5050-4c8f-990b-fa4824ff57b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:38:35.351680 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47571ce9-5050-4c8f-990b-fa4824ff57b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:38:35.351959 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47571ce9-5050-4c8f-990b-fa4824ff57b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:38:35.360936 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47571ce9-5050-4c8f-990b-fa4824ff57b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:38:35.362397 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47571ce9-5050-4c8f-990b-fa4824ff57b3 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:38:35.368351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:38:35.373787 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1441/2871] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:38:35 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:39:18.021710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777421d8-2dd8-4b4a-8c4c-5436637cbe5a None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:39:18.023240 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-777421d8-2dd8-4b4a-8c4c-5436637cbe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:39:18.085173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777421d8-2dd8-4b4a-8c4c-5436637cbe5a None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:39:18.088757 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea23ae-3d62-4f69-9b4b-a56440006d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:39:18.089399 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea23ae-3d62-4f69-9b4b-a56440006d66 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:39:18.089683 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea23ae-3d62-4f69-9b4b-a56440006d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:39:18.089959 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea23ae-3d62-4f69-9b4b-a56440006d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:39:18.091834 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5eea23ae-3d62-4f69-9b4b-a56440006d66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:39:18.165736 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea23ae-3d62-4f69-9b4b-a56440006d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:39:18.167262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea23ae-3d62-4f69-9b4b-a56440006d66 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:39:18.179191 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1431/2872] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:39:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:39:19.363136 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47571ce9-5050-4c8f-990b-fa4824ff57b3 service cinder] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:39:19.373639 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47571ce9-5050-4c8f-990b-fa4824ff57b3 service cinder] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:39:19.377213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ebd8b-a929-4b8d-b969-9d0a941e8fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:39:19.377848 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ebd8b-a929-4b8d-b969-9d0a941e8fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:39:19.378134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ebd8b-a929-4b8d-b969-9d0a941e8fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:39:19.378430 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ebd8b-a929-4b8d-b969-9d0a941e8fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:39:19.380213 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-5b5ebd8b-a929-4b8d-b969-9d0a941e8fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:39:19.393930 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ebd8b-a929-4b8d-b969-9d0a941e8fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:39:19.395404 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ebd8b-a929-4b8d-b969-9d0a941e8fd1 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:39:19.407012 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1442/2873] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:39:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:39:19.706159 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea23ae-3d62-4f69-9b4b-a56440006d66 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:39:19.715696 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea23ae-3d62-4f69-9b4b-a56440006d66 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:39:19.719403 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c88ba-ae0b-4ac9-ab97-0cda081ad500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:39:19.720041 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c88ba-ae0b-4ac9-ab97-0cda081ad500 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:39:19.720331 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c88ba-ae0b-4ac9-ab97-0cda081ad500 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:39:19.720609 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c88ba-ae0b-4ac9-ab97-0cda081ad500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:39:19.722389 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-570c88ba-ae0b-4ac9-ab97-0cda081ad500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:39:19.786913 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c88ba-ae0b-4ac9-ab97-0cda081ad500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:39:19.788360 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c88ba-ae0b-4ac9-ab97-0cda081ad500 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:39:19.798015 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1432/2874] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:39:19 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:39:28.182961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ebd8b-a929-4b8d-b969-9d0a941e8fd1 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:39:28.245518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ebd8b-a929-4b8d-b969-9d0a941e8fd1 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:39:28.250011 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:39:28.250928 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:39:28.251212 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:39:28.251490 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] PATH_INFO: `/v3/users/f3fef41cb8b54f0bbc62c57995475fb8` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:39:28.252658 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3fef41cb8b54f0bbc62c57995475fb8)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:39:28.254596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:39:28.436665 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3fef41cb8b54f0bbc62c57995475fb8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:28.443916 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3fef41cb8b54f0bbc62c57995475fb8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:28.444587 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3fef41cb8b54f0bbc62c57995475fb8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:28.445208 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3fef41cb8b54f0bbc62c57995475fb8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:28.513755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3fef41cb8b54f0bbc62c57995475fb8'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:28.531755 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1443/2875] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:39:28 2022] DELETE /identity/v3/users/f3fef41cb8b54f0bbc62c57995475fb8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:39:28.536503 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce032b6-41e9-4b26-a823-2b56c1942567 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:39:28.536985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce032b6-41e9-4b26-a823-2b56c1942567 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:39:28.537101 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce032b6-41e9-4b26-a823-2b56c1942567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:39:28.554292 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-5ce032b6-41e9-4b26-a823-2b56c1942567 None None] MFA Rules not processed for user `87e06b5266a1425fb48c02407f951f73`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:39:28.555794 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5ce032b6-41e9-4b26-a823-2b56c1942567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:39:28.585769 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=100 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:28.613836 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1433/2876] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:39:28.623933 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:39:28.625326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:39:28.684020 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404fc90-6eee-4032-bccf-b7e76a5c2129 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:39:28.687730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5c9fe-2283-4425-9657-67bd2caf4c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:39:28.688322 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5c9fe-2283-4425-9657-67bd2caf4c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:39:28.688611 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5c9fe-2283-4425-9657-67bd2caf4c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:39:28.688893 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5c9fe-2283-4425-9657-67bd2caf4c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:39:28.699257 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5c9fe-2283-4425-9657-67bd2caf4c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:39:28.701555 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5c9fe-2283-4425-9657-67bd2caf4c88 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:39:28.714298 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1444/2877] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:39:29.133310 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce032b6-41e9-4b26-a823-2b56c1942567 None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:39:29.136656 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-5ce032b6-41e9-4b26-a823-2b56c1942567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:39:29.146798 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce032b6-41e9-4b26-a823-2b56c1942567 None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:39:29.151641 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:39:29.153234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:39:29.153234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:39:29.153234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] PATH_INFO: `/v3/projects/ecbc0d593e974432a4aceaf6044460bd` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:39:29.154241 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecbc0d593e974432a4aceaf6044460bd)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:39:29.156183 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:39:29.264083 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecbc0d593e974432a4aceaf6044460bd', 'request_id': 'req-d780650f-c8ea-4684-88be-350691d8513d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:29.269572 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecbc0d593e974432a4aceaf6044460bd', 'request_id': 'req-d780650f-c8ea-4684-88be-350691d8513d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:29.348544 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecbc0d593e974432a4aceaf6044460bd', 'request_id': 'req-d780650f-c8ea-4684-88be-350691d8513d'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:29.360712 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] The token cache is being invalidate because project ecbc0d593e974432a4aceaf6044460bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:39:29.360996 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:29.361615 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:39:29.363175 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1434/2878] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:39:29 2022] DELETE /identity/v3/projects/ecbc0d593e974432a4aceaf6044460bd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:39:38.714711 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:39:39.357682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:08.967727 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96557f05-90b0-45d4-b111-5e3d216c36e6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:40:08.968211 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96557f05-90b0-45d4-b111-5e3d216c36e6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:40:08.968321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96557f05-90b0-45d4-b111-5e3d216c36e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:40:08.987436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-96557f05-90b0-45d4-b111-5e3d216c36e6 None None] MFA Rules not processed for user `0beb4f30b5c34ab0afdbfbf46dbfede5`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:40:08.988646 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-96557f05-90b0-45d4-b111-5e3d216c36e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:40:09.044979 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1445/2879] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:40:12.974346 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:40:12.975916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:40:13.044861 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780650f-c8ea-4684-88be-350691d8513d admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:40:13.051785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec3ff2-9f95-41bb-99f0-5dc1f2a98a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:40:13.051785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3ff2-9f95-41bb-99f0-5dc1f2a98a2e service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:40:13.051785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3ff2-9f95-41bb-99f0-5dc1f2a98a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:40:13.051785 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3ff2-9f95-41bb-99f0-5dc1f2a98a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:40:13.052155 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-f0ec3ff2-9f95-41bb-99f0-5dc1f2a98a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:40:13.061670 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec3ff2-9f95-41bb-99f0-5dc1f2a98a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:40:13.063122 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec3ff2-9f95-41bb-99f0-5dc1f2a98a2e service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:40:13.071447 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1435/2880] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:40:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:40:19.038237 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=25 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:23.067820 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=26 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:40:31.106251 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96557f05-90b0-45d4-b111-5e3d216c36e6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:40:31.107717 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-96557f05-90b0-45d4-b111-5e3d216c36e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:40:31.188078 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96557f05-90b0-45d4-b111-5e3d216c36e6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:40:31.194351 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce714b-657a-4022-9fd5-46a48fb29290 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:40:31.195077 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce714b-657a-4022-9fd5-46a48fb29290 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:40:31.195366 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce714b-657a-4022-9fd5-46a48fb29290 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:40:31.195649 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce714b-657a-4022-9fd5-46a48fb29290 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:40:31.197551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-65ce714b-657a-4022-9fd5-46a48fb29290 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:40:31.211082 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce714b-657a-4022-9fd5-46a48fb29290 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:40:31.212743 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce714b-657a-4022-9fd5-46a48fb29290 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:40:31.223762 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1446/2881] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:40:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:40:41.218897 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=34 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:41:49.730964 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec3ff2-9f95-41bb-99f0-5dc1f2a98a2e service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:41:49.739493 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec3ff2-9f95-41bb-99f0-5dc1f2a98a2e service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:41:49.744359 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4048c-8942-4854-ae0f-90df81a85ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:41:49.745065 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4048c-8942-4854-ae0f-90df81a85ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:41:49.745344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4048c-8942-4854-ae0f-90df81a85ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:41:49.745639 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4048c-8942-4854-ae0f-90df81a85ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:41:49.747463 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-6cd4048c-8942-4854-ae0f-90df81a85ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:41:49.757800 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4048c-8942-4854-ae0f-90df81a85ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:41:49.759265 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4048c-8942-4854-ae0f-90df81a85ea0 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:41:49.767481 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1436/2882] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:41:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:41:50.108970 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36ea4-8ba6-47a6-8ddb-521b91258a69 None None] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:41:50.111961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36ea4-8ba6-47a6-8ddb-521b91258a69 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:41:50.111961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36ea4-8ba6-47a6-8ddb-521b91258a69 None None] PATH_INFO: `/` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:41:50.111961 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1447/2883] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:41:50.117731 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d197ee-3860-4929-8295-f6da3c74ea74 None None] REQUEST_METHOD: `POST` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:41:50.118017 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d197ee-3860-4929-8295-f6da3c74ea74 None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:41:50.118299 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d197ee-3860-4929-8295-f6da3c74ea74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:41:50.164726 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.auth.core [None req-b0d197ee-3860-4929-8295-f6da3c74ea74 None None] MFA Rules not processed for user `8b266ea9964c4a179c2743b9cf6478eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:41:50.165952 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-b0d197ee-3860-4929-8295-f6da3c74ea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:41:50.172725 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1437/2884] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:41:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:41:50.186066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36ea4-8ba6-47a6-8ddb-521b91258a69 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:41:50.187824 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-43f36ea4-8ba6-47a6-8ddb-521b91258a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:41:50.200948 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36ea4-8ba6-47a6-8ddb-521b91258a69 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:41:50.204436 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e119c5-d28f-4138-bd87-c168df7c2dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:41:50.205066 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e119c5-d28f-4138-bd87-c168df7c2dfe service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:41:50.205352 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e119c5-d28f-4138-bd87-c168df7c2dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:41:50.205628 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e119c5-d28f-4138-bd87-c168df7c2dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:41:50.212832 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e119c5-d28f-4138-bd87-c168df7c2dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:41:50.214241 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e119c5-d28f-4138-bd87-c168df7c2dfe service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:41:50.222182 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1448/2885] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:41:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:41:50.307401 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e016-dc03-4e2b-ba20-4b56f078afae None None] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:41:50.307723 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e016-dc03-4e2b-ba20-4b56f078afae None None] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:41:50.308008 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e016-dc03-4e2b-ba20-4b56f078afae None None] PATH_INFO: `/` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:41:50.309374 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1438/2886] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 19 12:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 12:41:50.314205 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889ece9-467e-469e-b57c-29046968eb4c None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:41:50.314530 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889ece9-467e-469e-b57c-29046968eb4c None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:41:50.314820 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889ece9-467e-469e-b57c-29046968eb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:41:50.351306 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-9889ece9-467e-469e-b57c-29046968eb4c None None] MFA Rules not processed for user `bb6776311f354b748d563c4f1d793972`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:41:50.352500 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9889ece9-467e-469e-b57c-29046968eb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:41:50.361671 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1449/2887] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 19 12:41:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:41:50.372586 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e016-dc03-4e2b-ba20-4b56f078afae None None] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:41:50.383896 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e016-dc03-4e2b-ba20-4b56f078afae None None] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:41:50.387535 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329336ac-2bb6-455f-b6ca-f0f3455fabdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:41:50.388173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329336ac-2bb6-455f-b6ca-f0f3455fabdb service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:41:50.388459 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329336ac-2bb6-455f-b6ca-f0f3455fabdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:41:50.388740 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329336ac-2bb6-455f-b6ca-f0f3455fabdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:41:50.404522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329336ac-2bb6-455f-b6ca-f0f3455fabdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:41:50.405953 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329336ac-2bb6-455f-b6ca-f0f3455fabdb service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:41:50.417236 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1439/2888] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:41:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:42:00.348417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=15 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:00.412427 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=25 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:19.762919 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889ece9-467e-469e-b57c-29046968eb4c None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:42:19.764292 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-9889ece9-467e-469e-b57c-29046968eb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:19.820130 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889ece9-467e-469e-b57c-29046968eb4c None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:42:19.823640 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ac7c9-f8fa-426a-8d33-0b1f1011c0c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '316ad1c2eccf4642a21c68856585ceee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:42:19.824287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ac7c9-f8fa-426a-8d33-0b1f1011c0c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:42:19.824573 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ac7c9-f8fa-426a-8d33-0b1f1011c0c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:42:19.824853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ac7c9-f8fa-426a-8d33-0b1f1011c0c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:42:19.826613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-308ac7c9-f8fa-426a-8d33-0b1f1011c0c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:19.837788 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ac7c9-f8fa-426a-8d33-0b1f1011c0c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:42:19.839238 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ac7c9-f8fa-426a-8d33-0b1f1011c0c9 service cinder] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:42:19.848005 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1450/2889] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:42:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:42:25.028367 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329336ac-2bb6-455f-b6ca-f0f3455fabdb service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:42:25.029336 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-329336ac-2bb6-455f-b6ca-f0f3455fabdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:25.097520 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329336ac-2bb6-455f-b6ca-f0f3455fabdb service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:42:25.102985 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba237b8a-b682-4fa2-9ca4-132557e0705a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:42:25.103632 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba237b8a-b682-4fa2-9ca4-132557e0705a service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:42:25.103917 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba237b8a-b682-4fa2-9ca4-132557e0705a service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:42:25.104195 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba237b8a-b682-4fa2-9ca4-132557e0705a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:42:25.105932 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-ba237b8a-b682-4fa2-9ca4-132557e0705a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:25.117974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba237b8a-b682-4fa2-9ca4-132557e0705a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:42:25.119588 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba237b8a-b682-4fa2-9ca4-132557e0705a service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:42:25.129334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1440/2890] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:42:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:42:29.843993 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=26 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:31.432287 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704707d0-20f0-4254-b76a-b0f9ff2fc1a6 None None] REQUEST_METHOD: `POST` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:42:31.432892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704707d0-20f0-4254-b76a-b0f9ff2fc1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:42:31.432892 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704707d0-20f0-4254-b76a-b0f9ff2fc1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:42:31.450551 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.auth.core [None req-704707d0-20f0-4254-b76a-b0f9ff2fc1a6 None None] MFA Rules not processed for user `0beb4f30b5c34ab0afdbfbf46dbfede5`. Rule list: `[]` (Enabled: `True`). {{(pid=80603) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 12:42:31.451807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-704707d0-20f0-4254-b76a-b0f9ff2fc1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:31.459028 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1451/2891] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 19 12:42:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 12:42:31.779228 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba237b8a-b682-4fa2-9ca4-132557e0705a service placement] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:42:31.788884 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba237b8a-b682-4fa2-9ca4-132557e0705a service placement] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:42:31.792342 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:42:31.792968 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:42:31.793250 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:42:31.793525 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:42:31.795282 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:31.805180 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:42:31.806882 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:42:31.814762 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1441/2892] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:42:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:42:34.578175 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704707d0-20f0-4254-b76a-b0f9ff2fc1a6 None None] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:42:34.579613 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-704707d0-20f0-4254-b76a-b0f9ff2fc1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:34.594336 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704707d0-20f0-4254-b76a-b0f9ff2fc1a6 None None] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:42:34.599344 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629221e0-9e78-4ed9-91eb-01ac8acaa3f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:42:34.599981 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629221e0-9e78-4ed9-91eb-01ac8acaa3f3 service placement] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:42:34.600266 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629221e0-9e78-4ed9-91eb-01ac8acaa3f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:42:34.600546 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629221e0-9e78-4ed9-91eb-01ac8acaa3f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:42:34.602235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-629221e0-9e78-4ed9-91eb-01ac8acaa3f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:34.613370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629221e0-9e78-4ed9-91eb-01ac8acaa3f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:42:34.615529 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629221e0-9e78-4ed9-91eb-01ac8acaa3f3 service placement] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:42:34.625829 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1452/2893] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:42:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:42:41.811234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=38 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:44.621178 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=9 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:42:52.952188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:42:52.952188 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:52.965818 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c330d08-ea32-414c-a8a9-d220416c2277 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:42:52.971350 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e9ba-3e04-4c2c-87ec-7379979bea28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:42:52.971984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e9ba-3e04-4c2c-87ec-7379979bea28 service nova] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:42:52.972269 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e9ba-3e04-4c2c-87ec-7379979bea28 service nova] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:42:52.972546 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e9ba-3e04-4c2c-87ec-7379979bea28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:42:52.974923 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-7350e9ba-3e04-4c2c-87ec-7379979bea28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:52.986906 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e9ba-3e04-4c2c-87ec-7379979bea28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:42:52.988470 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e9ba-3e04-4c2c-87ec-7379979bea28 service nova] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:42:52.999489 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1442/2894] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:42:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:42:55.742778 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629221e0-9e78-4ed9-91eb-01ac8acaa3f3 service placement] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:42:55.751964 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629221e0-9e78-4ed9-91eb-01ac8acaa3f3 service placement] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:42:55.756739 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d14708-2552-4dd6-8275-71e57de6e969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:42:55.756900 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d14708-2552-4dd6-8275-71e57de6e969 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:42:55.757065 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d14708-2552-4dd6-8275-71e57de6e969 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:42:55.757345 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d14708-2552-4dd6-8275-71e57de6e969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:42:55.759132 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-19d14708-2552-4dd6-8275-71e57de6e969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:42:55.823480 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d14708-2552-4dd6-8275-71e57de6e969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:42:55.824918 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d14708-2552-4dd6-8275-71e57de6e969 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:42:55.834453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1453/2895] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:42:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:43:02.995054 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:05.830123 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=30 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:44.242574 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e9ba-3e04-4c2c-87ec-7379979bea28 service nova] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:43:44.252013 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e9ba-3e04-4c2c-87ec-7379979bea28 service nova] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:43:44.255570 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c358be-cad6-4208-b602-4348cbea3495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:43:44.256203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c358be-cad6-4208-b602-4348cbea3495 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:43:44.256487 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c358be-cad6-4208-b602-4348cbea3495 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:43:44.256763 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c358be-cad6-4208-b602-4348cbea3495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:43:44.258539 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-d3c358be-cad6-4208-b602-4348cbea3495 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:43:44.317206 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c358be-cad6-4208-b602-4348cbea3495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:43:44.318668 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c358be-cad6-4208-b602-4348cbea3495 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:43:44.327707 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1443/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:43:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:43:45.230288 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d14708-2552-4dd6-8275-71e57de6e969 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:43:45.242865 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d14708-2552-4dd6-8275-71e57de6e969 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:43:45.249303 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb6776311f354b748d563c4f1d793972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:43:45.254308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:43:45.254308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:43:45.254308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:43:45.254308 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:43:45.267272 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:43:45.269196 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:43:45.282709 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1454/2897] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:43:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:43:50.625597 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c358be-cad6-4208-b602-4348cbea3495 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:43:50.635063 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c358be-cad6-4208-b602-4348cbea3495 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:43:50.638715 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e5513-31f2-4210-ad6a-662cf7605b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:43:50.639348 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5513-31f2-4210-ad6a-662cf7605b46 service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:43:50.639631 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5513-31f2-4210-ad6a-662cf7605b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:43:50.639908 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e5513-31f2-4210-ad6a-662cf7605b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:43:50.641722 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-686e5513-31f2-4210-ad6a-662cf7605b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:43:50.703804 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e5513-31f2-4210-ad6a-662cf7605b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:43:50.705234 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e5513-31f2-4210-ad6a-662cf7605b46 service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:43:50.715380 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1444/2898] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:43:55.278234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=6 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:58.050328 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:43:58.051868 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:43:58.066036 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16075c-5f3f-4411-86df-919f372d4d96 service nova] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:43:58.070526 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:43:58.071514 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:43:58.071799 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:43:58.072076 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] PATH_INFO: `/v3/users/2273aeca59c04c778683e70665b7efa1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:43:58.073789 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2273aeca59c04c778683e70665b7efa1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:43:58.075734 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:43:58.244462 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2273aeca59c04c778683e70665b7efa1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.251203 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2273aeca59c04c778683e70665b7efa1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.252730 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2273aeca59c04c778683e70665b7efa1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.253331 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2273aeca59c04c778683e70665b7efa1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.336539 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2273aeca59c04c778683e70665b7efa1'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.355222 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1455/2899] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:43:58 2022] DELETE /identity/v3/users/2273aeca59c04c778683e70665b7efa1 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:43:58.600556 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e5513-31f2-4210-ad6a-662cf7605b46 service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:43:58.611016 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e5513-31f2-4210-ad6a-662cf7605b46 service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:43:58.669344 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:43:58.670203 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:43:58.670515 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:43:58.670797 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] PATH_INFO: `/v3/projects/d3a4bb3cecc4474e85a49a217fa98d9f` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:43:58.671989 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a4bb3cecc4474e85a49a217fa98d9f)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:43:58.673909 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:43:58.774387 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:58.783519 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a4bb3cecc4474e85a49a217fa98d9f', 'request_id': 'req-0055a546-3d02-4f13-a19a-d5586bb7458c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.788872 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a4bb3cecc4474e85a49a217fa98d9f', 'request_id': 'req-0055a546-3d02-4f13-a19a-d5586bb7458c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.867916 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a4bb3cecc4474e85a49a217fa98d9f', 'request_id': 'req-0055a546-3d02-4f13-a19a-d5586bb7458c'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.880181 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] The token cache is being invalidate because project d3a4bb3cecc4474e85a49a217fa98d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:43:58.880466 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.881174 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:58.882758 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1445/2900] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:43:58 2022] DELETE /identity/v3/projects/d3a4bb3cecc4474e85a49a217fa98d9f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:43:58.887154 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:43:58.888533 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:43:58.901474 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f15e6b-ede5-4bd6-a42c-f4fc75bcba54 admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:43:58.956956 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:43:58.957816 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:43:58.958097 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:43:58.958370 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] PATH_INFO: `/v3/users/7f6f34a427aa4f348afb98a6b5987ef5` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:43:58.959559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6f34a427aa4f348afb98a6b5987ef5)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:43:58.961455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:43:59.098884 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6f34a427aa4f348afb98a6b5987ef5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.106399 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6f34a427aa4f348afb98a6b5987ef5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.107092 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6f34a427aa4f348afb98a6b5987ef5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.107716 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6f34a427aa4f348afb98a6b5987ef5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.153675 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6f34a427aa4f348afb98a6b5987ef5'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.161666 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:43:59.176286 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1456/2901] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:43:58 2022] DELETE /identity/v3/users/7f6f34a427aa4f348afb98a6b5987ef5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:43:59.440878 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:43:59.442334 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:43:59.455891 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055a546-3d02-4f13-a19a-d5586bb7458c admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:43:59.514255 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:43:59.515156 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:43:59.515438 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:43:59.515719 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] PATH_INFO: `/v3/projects/99d9ea0ac1794cee8e7651632c0e88dd` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:43:59.516904 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d9ea0ac1794cee8e7651632c0e88dd)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:43:59.518890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:43:59.639557 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d9ea0ac1794cee8e7651632c0e88dd', 'request_id': 'req-0cd73aa1-8c3b-4242-a422-57115f4bd423'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.645682 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d9ea0ac1794cee8e7651632c0e88dd', 'request_id': 'req-0cd73aa1-8c3b-4242-a422-57115f4bd423'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.734078 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d9ea0ac1794cee8e7651632c0e88dd', 'request_id': 'req-0cd73aa1-8c3b-4242-a422-57115f4bd423'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.746185 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] The token cache is being invalidate because project 99d9ea0ac1794cee8e7651632c0e88dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:43:59.746543 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.747173 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:43:59.748705 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1446/2902] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:43:59 2022] DELETE /identity/v3/projects/99d9ea0ac1794cee8e7651632c0e88dd => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:44:09.279742 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:09.743106 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:47.666722 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:44:47.668209 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:44:47.728548 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cd20b-e6a3-4f31-bb19-357d958fe27b admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:44:47.781531 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:44:47.782234 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:44:47.782559 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:44:47.782840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:44:47.784596 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:44:47.863720 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:44:47.865179 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:44:47.878453 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1457/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:44:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:44:49.064074 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:44:49.065547 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:44:49.078469 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd73aa1-8c3b-4242-a422-57115f4bd423 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:44:49.082020 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:44:49.082690 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:44:49.082974 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:44:49.083249 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:44:49.084935 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:44:49.098877 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:44:49.100279 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:44:49.111086 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1447/2904] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:44:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:44:57.873321 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=74 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:44:59.106453 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=6 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:45:00.973173 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:45:00.975518 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:45:00.986917 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c9d45-39cf-4bac-9b6c-8d31d2036aa5 service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:45:00.990643 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969025b8-cba3-432f-9e6d-95456065247a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b266ea9964c4a179c2743b9cf6478eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:45:00.991283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969025b8-cba3-432f-9e6d-95456065247a service neutron] REQUEST_METHOD: `GET` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:45:00.991565 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969025b8-cba3-432f-9e6d-95456065247a service neutron] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:45:00.991840 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969025b8-cba3-432f-9e6d-95456065247a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:45:00.993581 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-969025b8-cba3-432f-9e6d-95456065247a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:45:01.059360 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969025b8-cba3-432f-9e6d-95456065247a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:45:01.060889 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969025b8-cba3-432f-9e6d-95456065247a service neutron] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:45:01.070455 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1458/2905] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:45:08.460890 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:45:08.462333 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:45:08.477812 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e40c9-dc5a-42b8-b476-6527a906b60a service neutron] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:45:08.483119 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:45:08.483982 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:45:08.484262 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:45:08.484534 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] PATH_INFO: `/v3/users/0beb4f30b5c34ab0afdbfbf46dbfede5` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:45:08.485704 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0beb4f30b5c34ab0afdbfbf46dbfede5)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:45:08.487662 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:45:08.662372 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0beb4f30b5c34ab0afdbfbf46dbfede5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:08.663107 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0beb4f30b5c34ab0afdbfbf46dbfede5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:08.675518 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0beb4f30b5c34ab0afdbfbf46dbfede5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:08.676165 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0beb4f30b5c34ab0afdbfbf46dbfede5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:08.757246 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0beb4f30b5c34ab0afdbfbf46dbfede5'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:08.770580 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1448/2906] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:45:08 2022] DELETE /identity/v3/users/0beb4f30b5c34ab0afdbfbf46dbfede5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:45:09.032134 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969025b8-cba3-432f-9e6d-95456065247a service neutron] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:45:09.040630 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969025b8-cba3-432f-9e6d-95456065247a service neutron] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:45:09.099085 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:45:09.099995 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:45:09.100283 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:45:09.100563 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] PATH_INFO: `/v3/projects/ff5ebee4cf6c47f9adee30f9c8cd6ba1` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:45:09.101729 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff5ebee4cf6c47f9adee30f9c8cd6ba1)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:45:09.103682 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:45:09.220513 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff5ebee4cf6c47f9adee30f9c8cd6ba1', 'request_id': 'req-2b584fb6-4214-46e7-9db9-534bf80f45ea'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.225921 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff5ebee4cf6c47f9adee30f9c8cd6ba1', 'request_id': 'req-2b584fb6-4214-46e7-9db9-534bf80f45ea'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.294473 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff5ebee4cf6c47f9adee30f9c8cd6ba1', 'request_id': 'req-2b584fb6-4214-46e7-9db9-534bf80f45ea'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.305807 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] The token cache is being invalidate because project ff5ebee4cf6c47f9adee30f9c8cd6ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:45:09.306096 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.306853 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.308938 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1459/2907] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:45:09 2022] DELETE /identity/v3/projects/ff5ebee4cf6c47f9adee30f9c8cd6ba1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:45:09.313139 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:45:09.314649 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:45:09.328509 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c592d6-bd98-4bb8-96d8-39c77fa54df8 admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:45:09.385194 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:45:09.386827 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:45:09.387112 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:45:09.387393 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] PATH_INFO: `/v3/users/09388632784a43eebbfb94bd3c0f29f6` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:45:09.388590 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=09388632784a43eebbfb94bd3c0f29f6)` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:45:09.390561 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:45:09.612293 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09388632784a43eebbfb94bd3c0f29f6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.612960 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09388632784a43eebbfb94bd3c0f29f6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.625637 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09388632784a43eebbfb94bd3c0f29f6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.626259 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09388632784a43eebbfb94bd3c0f29f6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.700415 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.notifications [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09388632784a43eebbfb94bd3c0f29f6'} {{(pid=80604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:09.702739 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:45:09.713522 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1449/2908] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 19 12:45:09 2022] DELETE /identity/v3/users/09388632784a43eebbfb94bd3c0f29f6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:45:09.959126 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] Authenticating user token {{(pid=80603) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:45:09.960593 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.fernet_utils [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80603) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:45:09.974815 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584fb6-4214-46e7-9db9-534bf80f45ea admin admin] Validating token access rules against request {{(pid=80603) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:45:09.996674 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:45:10.035417 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87e06b5266a1425fb48c02407f951f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67ffb05dd43a48979db84b48835b3e07', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80603) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:45:10.036291 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:45:10.036577 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:45:10.036910 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] PATH_INFO: `/v3/projects/b46af4302be94737a202b2f1d3664547` {{(pid=80603) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:45:10.038089 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b46af4302be94737a202b2f1d3664547)` {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:45:10.040094 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] RBAC: Authorization granted {{(pid=80603) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:45:10.126660 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b46af4302be94737a202b2f1d3664547', 'request_id': 'req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:10.132235 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b46af4302be94737a202b2f1d3664547', 'request_id': 'req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:10.185719 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b46af4302be94737a202b2f1d3664547', 'request_id': 'req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9'} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:10.197033 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] The token cache is being invalidate because project b46af4302be94737a202b2f1d3664547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80603) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 12:45:10.197326 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:10.197990 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG keystone.notifications [None req-5c5ecaca-71de-4a9e-b1b4-e8bf8b3984f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80603) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 12:45:10.199557 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: [pid: 80603|app: 0|req: 1460/2909] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 19 12:45:09 2022] DELETE /identity/v3/projects/b46af4302be94737a202b2f1d3664547 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 12:45:19.877030 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=4 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:45:20.193776 dv-cinder-778193-27-iscsi devstack@keystone.service[80603]: DEBUG dbcounter [-] [80603] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80603) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 12:47:30.066009 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] Authenticating user token {{(pid=80604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 12:47:30.069347 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:47:30.209133 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf8051-8331-4939-b3f1-926d86d09e7f admin admin] Validating token access rules against request {{(pid=80604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 12:47:30.267779 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2448b5-a438-4656-906a-3560de7ce701 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f41920bad3147bba0000de911dfe5a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94ce53bfef52443580430ddb1eb6185a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 12:47:30.268429 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2448b5-a438-4656-906a-3560de7ce701 service placement] REQUEST_METHOD: `GET` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 12:47:30.268710 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2448b5-a438-4656-906a-3560de7ce701 service placement] SCRIPT_NAME: `/identity` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 12:47:30.268984 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2448b5-a438-4656-906a-3560de7ce701 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 12:47:30.270829 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.fernet_utils [None req-fb2448b5-a438-4656-906a-3560de7ce701 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 12:47:30.282455 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2448b5-a438-4656-906a-3560de7ce701 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 12:47:30.284032 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2448b5-a438-4656-906a-3560de7ce701 service placement] RBAC: Authorization granted {{(pid=80604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 12:47:30.293694 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: [pid: 80604|app: 0|req: 1450/2910] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 19 12:47:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 12:47:40.289508 dv-cinder-778193-27-iscsi devstack@keystone.service[80604]: DEBUG dbcounter [-] [80604] Writing DB stats keystone:SELECT=89 {{(pid=80604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}